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Abstract 

Temporal  projection,  defined  as  the  prediction  of  what  might  happen  when  a  plan  is  executed, 
is  an  important  component  of  many  planning  algorithms.  To  achieve  efficiency,  it  is  desirable  for 
a  projection  to  be  a  totally  ordered  event  sequence.  To  cope  with  uncertainty,  the  events  must  be 
generated  using  probabilistic  rules.  We  require  a  rule  language  that  allows  us  to  specify  what  can 
happen  when  an  event  occurs,  as  well  as  what  events  can  occur  when  certain  propositions  are  true. 
The  language  has  a  formal  semantics,  which  allows  us  to  prove  that  a  set  of  rules  has  a  unique  model 
(if  it  is  “consistent”).  This  language  supports  a  Monte  Carlo  style  of  projection,  in  which  event 
sequences  are  sampled  randomly  using  the  probabilities  in  the  rules.  The  output  of  the  projector  is 
a  timeline  that  allows  a  planning  algorithm  to  test  the  truth  of  propositions  at  arbitrary  points.  The 
algorithms  for  building  and  retrieving  from  the  timeline  can  be  shown  to  be  correct.  Experiments 
show  that  for  a  typical  theory,  the  time  to  build  a  timeline  is  a  quadratic  function  of  the  number  of 
events  in  the  timeline. 


1  The  Problem 


Automated  planners  need  to  do  temporal  reasoning,  that  is,  to  decide  what  will  be  true  at  various  times 
if  their  plans  are  executed,  in  support  of  planning  operations  (such  as  reordering  plan  steps)  that  depend 
on  when  various  facts  become  true  or  false  during  plan  execution.  The  main  line  of  research  in  this  area 
is  to  represent  a  plan  as  a  partially  ordered  list  of  events  (more  precisely,  as  a  totally  ordered  set  of  events 
with  some  of  the  orderings  unknown),  and  to  attempt  to  infer  what  must  be  true  before  or  after  each 
event.  In  many  such  efforts,  it  is  assumed  that  the  events’  effects  are  all  known  and  context-independent, 
so  that  the  fact  P  is  true  after  event  e  if  and  only  if  there  is  some  event  e'  preceding  or  coinciding  with 
e  that  has  P  as  an  effect,  and  such  that  for  every  event  e"  with  -> P  as  an  effect,  e"  precedes  e'  or 
follows  e.  (McAllester  and  Rosenblitt  1991,  Dean  and  McDermott  1987,  Chapman  1987,  Allen  1984, 
VanBeek  1992) 

“This  work  was  supported  by  the  Advanced  Research  Projects  Agency  of  the  Defense  Department,  under  ONR  Contract 
Number  N00014-91- J-1S77. 
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However,  in  general  the  effects  of  an  event  depend  on  what  is  true  before  the  event,  and  in  this  more 
general  setting  the  problem  becomes  much  harder  (Dean  and  Boddy  1988).  One  way  to  cope  with  the 
complexity  is  to  stop  trying  to  work  with  a  partial  ordering  representing  a  set  of  possible  total  orderings. 
Instead,  decouple  the  plan  from  the  temporal  database  completely.  Given  a  plan,  we  can  generate  several 
projections  of  the  plan,  each  of  which  is  a  totally  ordered  possible  execution  scenario.  Now,  instead  of 
asking,  What  must  be  true  in  all  projections?,  we  can  ask,  What  is  true  in  the  sample  of  projections 
generated  so  far?  There  are  several  advantages  to  this  approach: 

1.  Plans  can  remain  partially  ordered.  In  fact,  they  can  become  arbitrary  programs,  so  long  as  it  is 
possible  to  predict  how  they  might  be  executed  at  some  level  of  detail. 

2.  The  inference  machinery  for  each  projection  can  be  fast  and  simple. 

3.  The  more  projections  the  planner  generates,  the  more  it  learns  about  the  current  plan.  Under  time 
pressure,  it  can  generate  fewer  projections. 

4.  Probabilistic  world  models  can  be  handled. 

In  this  paper,  I  describe  an  algorithm  for  probabilistic,  totally-ordered  database  management.  The 
planning  algorithm  this  fits  into  is  described  in  (McDermott  1992).  (The  temporal  system  described 
in  that  paper  differs  in  detail  from  the  version  described  here.)  I  will  have  nothing  to  say  about  the 
overall  planning  architecture.  I  will  just  assume  that  there  is  a  projection  module  that  takes  a  plan  and 
produces  a  sequence  of  dated  events.  Each  event  occurs  at  a  single  time  instant.  (Events  with  duration 
are  modeled  as  beginning  at  one  instant  and  ending  at  another.)  The  job  of  the  temporal  inference 
system  is  to  build  a  data  structure  called  a  timeline  that  records  the  events  and  their  effects.  It  starts 
with  a  set  of  initial  conditions,  that  is,  facts  true  before  the  projection  begins.  At  any  point  before  or 
after  the  event  sequence  is  complete,  we  must  be  able  to  give  the  system  a  query  and  a  time  point,  and  it 
will  return  a  list  of  all  instances  of  the  query  that  are  true  at  that  point.  For  a  given  timeline,  it  always 
returns  the  same  answer  for  a  given  query  and  time  point,  but  exactly  which  answer  is  determined  by 
the  probabilities  in  the  laws  of  physics.  That  is,  if  a  new  timeline  were  constructed  from  the  same  event 
sequence,  the  same  query  could  give  different  results. 

This  is  not  the  first  work  in  the  area  of  probabilistic  projection.  The  formalism  I  develop  here  takes 
off  from  the  work  of  Hanks  (1990a,  1990b, Hanks  and  McDermott  1994),  who  developed  a  theory  of 
temporal  representation,  plus  an  algorithm  that  generated  all  possible  timelines  with  their  associated 
probabilities.  The  present  work  achieves  more  representational  power  at  the  cost  of  finding  only  ap¬ 
proximations  to  the  true  probabilities  of  different  outcomes.  Other  works,  notably  Kanazawa  (Dean 
and  Kanazawa  1989,  Kanazawa  1992),  have  reduced  probabilistic  temporal  reasoning  to  reasoning  about 
Bayes  nets  (Pearl  1988)  whose  nodes  represent  the  occurrence  of  different  events  and  propositions  at 
different  times.  Unfortunately,  the  resulting  nets  are  not  particularly  manageable.  The  present  work 
may  be  thought  of  as  an  application  of  stochastic  simulation  to  a  specialized  Bayes  net,  extended  to 
allow  for  the  generation  of  spontaneous  random  events.  However,  my  goal  is  not  to  evaluate  probabilities 
accurately,  but  instead  to  generate  plausible  scenarios  for  plan  execution.  It  may  well  be  more  valu¬ 
able  for  plan  debugging  to  be  able  to  inspect  samples  of  actual  causal  chains  than  to  have  an  accurate 
assessment  of  probabilities. 

There  have  been  several  other  stabs  at  formalizing  and  carrying  out  probabilistic  temporal  reasoning. 
Thiebaux  and  Hertzberg  (1992)  have  a  formalism  similar  to  mine  (more  expressive  in  some  ways,  less 
expressive  in  others),  that  allows  them  to  treat  plans  as  Markov  processes.  See  also  (Haddawy  1990). 
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2  An  Idealized  System 


In  a  temporal  setting,  propositions  are  not  merely  true  or  false.  They  become  true  at  a  time  point, 
persist  for  a  while,  become  false  at  a  time  point,  and  so  forth.  I  use  the  word  occasion  to  denote  a 
stretch  of  time  over  which  a  proposition  is  true.1  When  an  occasion  becomes  false,  it  is  said  to  be 
clipped.  Each  occasion  corresponds  to  an  atomic  formula,  such  as  loc(bozl,coorda(5,6)),3  called  the 
proposition  of  the  occasion.  Negation,  variables,  and  such  are  not  allowed  as  occasion  propositions. 

There  are  several  entry  points  to  the  system: 

1.  START-TIMELIIE:  Creates  a  new  timeline. 

2.  TIMELIIE-ADVAICE:  Adds  new  exogenous  events  to  the  timeline.  They  are  called  “exogenous” 
because  from  the  point  of  view  of  the  timeline  manager,  they  are  arbitrary  and  unmodeled.  In  the 
course  of  adding  a  new  exogenous  event,  it  may  randomly  spawn  some  endogenous  events,  that  is, 
events  whose  occurrence  is  modeled  by  rules  in  the  timeline  manager’s  model. 

3.  TIMELIIE-RETRIEVE:  Used  to  infer  what’s  true  at  the  end  of  the  timeline.  Both  TIMELIIE-ADVAICE 
and  TIMELIIE-RETRIEVE  use  an  internal  routine  called  TL-RETRIEVE,  which  can  determine  what’s 
true  at  an  arbitrary  point  in  the  timeline. 

4.  PAST-RETRIEVE:  Direct  entry  point  to  TL-RETRIEVE. 


All  these  routines  make  and  record  random  choices,  based  on  the  probabilities  in  the  rules  described 
below. 

It  is  important  to  realize  that  in  this  paper  I  am  speaking  only  of  the  temporal-reasoning  system, 
not  the  overall  planning  system  it  is  a  part  of  (described  inMcDermott  1992).  Some  of  the  ways  the 
narrower  system  is  described  will  seem  parochical  or  warped.  For  example,  the  labels  “exogenous”  and 
“endogenous”  applied  to  events  seem  backwards  from  the  point  of  view  of  the  overall  planner.  What 
I  call  “exogenous”  events  are  those  due  to  the  planner’s  own  actions,  while  “endogenous”  events  are 
the  autonomous  events  that  occur  in  the  part  of  the  world  outside  the  planner’s  control.  However,  even 
though  they  are  outside  the  agent’s  control,  there  is  a  probabilistic  model  of  them,  so  the  timeline  system 
knows  how  to  generate  them;  it  has  no  model  of  the  agent’s  actions. 

Another  possible  point  of  confusion  is  that  the  timelines  generated  by  the  projection  system  are  totally 
ordered,  while  the  plans  being  projected  are  not.  For  example,  our  plan  language,  called  RPL,  allows 
for  loops  and  parallel  combinations  of  events.  The  plan  (I-TIMES  K  (PAR  (P)  (Q)))  specifies  doing  P 
and  q  each  K  times,  in  no  particular  order  on  each  iteration.  K  is  a  RPL  variable,  whose  value  can  vary 
from  execution  to  execution.  A  typical  projection,  with  K=2,  might  look  like 

q.i.A  — q.i.fi  — *p .  i .  A  — q.i  .z  — p.i  .z  —p.2  .A  — q.2  .A  — q.2  .z  — p.2  .z 

'In  Dean  and  McDermott  1987,  these  were  called  “time  tokens.”  The  implementation  of  occasions  is  not  quite  faithful 
to  this  formal  definition,  as  we  will  see  in  Section  3. 

2  In  the  actual  implementation,  formulas  and  rules  are  expressed  in  a  Lispish  notation,  which  will  be  described  in 
Section  3. 
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where  l.i.Y  is  an  event  in  the  i’th  execution  of  X.  Events  are  labeled  A,  B,  ...,  Z .  For  instance,  the 
first  execution  of  Q  has  events  Q .  1 .  A  — ►£} .  1 .  B  — >Q .  1 .  Z  The  events  of  P .  1  are  interleaved  with  the 
events  of  Q.l,  and  similarly  for  P.2  and  Q.2.  But  although  the  interleavings  differ  from  projection 
to  projection,  for  any  given  projection  the  events  are  totally  ordered.  Finally,  in  this  paper  I  do  not 
explain  how  the  action  P  generates  events  like  P.  1  .A  and  P.l.Z  I’ll  just  take  events  as  given.  Consult 
(McDermott  1992)  for  the  linkage. 


2.1  Probabilistic  Rules 

There  are  several  forms  of  rule  for  expressing  the  “laws  of  physics”  in  a  given  domain: 
prob 

•  precond / event  |— ►.  effect:  Whenever  an  event  of  the  form  event  occurs,  when  precond  is  true, 
then,  with  probability  prob,  create  and  clip  occasions  as  specified  by  effect.  Used  whene'.rr  an 
event  is  added.  The  effect  can  be  a  conjunction  of  atomic  formulas  and  expressions  of  the  form 
->A,  where  A  is  an  atomic  formula.  The  ->  symbol  in  an  effect  is  not  interpreted  as  ordinary 
negation,  but  means  that  an  occasion  of  A  comes  to  an  end  with  this  event.  We  can  read  ->/  as 
“clip  A." 

prob 

•  effect  |  event\precond:  Same  meaning  as  |— but  used  to  answer  a  query  about  effect. 

elap 

•  precond  >|  event:  Over  any  interval  where  precond  is  true,  generates  random  endogenous  events, 
“Poisson-distributed,”  with  an  average  spacing  of  elap  time  units.  (The  exact  nature  of  the  distri¬ 
bution  is  explained  below.) 

•  p.*~.q:  If  q  is  true  at  a  time  point,  so  is  p.  Used  to  answer  queries  about  p.  p  is  an  atomic  formula; 
q  is  a  conjunction  of  literals. 

Rules  with  connective  |— ►  _  or  _  — *j  are  called  forward- chaining  rules.  Rule  with  connective  _  *— |  or 

-  are  called  backward-chaining  rules.  The  consequent  of  a  rule  is  the  part  on  the  same  side  of  the 
connective  as  the  arrowhead.  The  part  on  the  other  side  (precond  or  q)  is  called  the  antecedent. 

Rules  and  queries  may  contain  variables,  and  the  implementation  uses  unification  to  match  queries  with 
rules,  but  in  the  technical  sections  of  this  paper  we  will  avoid  thinking  of  the  system  as  being  a  full 
first-order  logic.  Instead,  I  will  treat  each  rule  as  a  schema  standing  for  all  its  ground  instances.  Because 
occasions  cannot  contain  variables,  we  impose  the  requirement  that  if  a  variable  occurs  in  the  consequent 
of  a  forward-chaining  rule,  then  it  must  also  occur  in  the  antecedent  of  that  rule,  or  in  its  event  if  it  has 
one.  Similarly,  if  a  variable  occurs  in  the  antecedent  of  a  backward-chaining  rule,  it  must  also  occur  in 
the  consequent  or  event.  And  if  a  variable  occurs  in  the  prob  of  a  |— *  _  or  the  elap  of  a  *|,  it  must 
occur  in  the  antecedent  (or  the  event  if  it’s  a  |— ►.);  if  it  appears  in  the  prob  of  a  |,  it  must  occur  in 
the  consequent. 

In  each  case,  the  antecedent  can  be  a  conjunction  of  literals,  that  is,  atomic  formulas  and  expressions 
of  the  form  ->A,  where  A  is  atomic.  The  empty  conjunction  can  be  written  true,  or  omitted.  As  I  will 
show,  this  system  is  simple  enough  that  ->  can  be  implemented  by  the  device  of  “negation  as  failure” 
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(Clark  1978).  Because  of  this,  I  impose  the  restriction  that  any  variable  that  occurs  in  a  negated  part 
of  an  antecedent  must  also  occur  in  an  unnegated  part. 

A  rule  instance  is  said  to  be  firablt  at  a  time  point  if  its  precondition  is  true  at  that  time  point.  It  is 
said  to  fin  if  it  is  Arable  and  its  probabilistic  test  comes  up  true.  The  probabilistic  test  for  a  given  firable 
rule  instance  is  performed  at  most  once  at  a  given  time  point.  (In  the  case  of  a  |  rule,  if  the  query 
never  occurs,  the  test  is  never  performed.) 

TIMELIfE-ADVAICE  uses  .  — *|  rules  to  add  endogenous  events  to  the  timeline.  It  calls  TL-RETRIEVE 
to  verify  the  preconditions  of  these  rules,  using  _  —  |  and  .  «—  _  rules.  For  every  event  added  by 
TIMELIIE-ADVAICE,  |— ».  rules  are  used  to  infer  the  consequences  of  the  event. 

Every  timeline  begins  with  an  event  of  the  form  START.  We  can  use  this  event  to  set  up  initial  conditions,3 
as  shown  in  the  following  example: 


-.Initially,  nuggstl  is  at  10,0,  and. . . 

1 

START  I—*.  loc(nuggetl,coords(10,0)) 

;T1000,  an  enemy  robot,  is  at  0,0 
1 

START  I—.  loc(T1000,coords(0,0)) 

.Roughly  every  10  seconds,  T1000  moves  right. 

10 

.-♦I  aova(T 1000, 1,0) 

;  When  an  agent  moves,  its  location  changes. 
loc(6,  coorda(x,  y))/mova(6,  Ax,  Ay) 

loc(6,  coords(x  +  Ax,  y  +  Ay))  A  ->loc(6,  coords(x,  y)) 

I  (We  don’t  nally  notate  addition  in  rules  this  way; 

,see  Section  3.) 

;  Whenever  T1000  and  the  nugget  an  in  the  same  place, 

;T1000  grasps  the  nugget,  typically  within  1  second. 

loc(T1000,  coords(x,  y))  A  loc(nuggetl,  coords(x,  V)) 

1 

.—►I  grasp(T1000, nugget  1) 

; Grasping  succeeds  with  a  probability  of  80%. 
holding(a,  6) 

0.8 

grasp(a,  6)\loc(a,  coords(x,  y))  A  loc(6,  coords(x,  y)) 

This  little  model  of  the  world  might  be  useful  if  the  planner  had  to  predict  the  chances  of  preventing  an 
enemy  robot  from  stealing  a  gold  nugget.  The  planner  could  project  the  plan  by  initializing  the  timeline, 
then  using  TIMELIIE-ADVAICE  to  add  the  agent’s  own  successive  actions  as  exogenous  events,  then  using 
TIMELIIE-RETRIEVE  with  query  loc(nugg«tl,  coords(10, 0))  to  find  out  if  the  nugget  is  still  at  10,0  at 
the  end.  The  enemy  will  take  about  100  seconds  to  get  to  10,0,  then  repeatedly  try  to  grasp  the  nugget, 
until  it  succeeds,  when  loc(nuggetl,  coords(10, 0))  will  be  clipped  and  grasping  will  cease.  Given  these 
rules,  the  only  way  to  stop  the  T1000  is  to  move  it  to  a  location  where  it  will  not  reach  the  nugget. 

3  There  are  more  flexible  ways  in  the  actual  implementation,  described  in  Section  3. 
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The  only  difference  between  j—  .  and  .  — |  rules  is  when  they  are  called.  .*—1  rules  have  the  advantage 
that  they  are  called  only  when  the  query  they  answer  arises.  1  call  the  application  of  — |  rules  “backward 
chaining,”  and  the  application  of  |— *.  rules  “forward  chaining”  by  analogy  with  their  static  counterparts. 
In  the  present  example,  there  was  really  no  reason  to  use  a  . «—  |  rule,  because  the  |— • .  rule  is  going 
to  require  querying  loc(nuggstl,  coords(. . .))  repeatedly  anyway.  This  pattern,  where  the  same  query 
is  repeated  at  many  time  points,  distinguishes  temporal  inference  from  routine  backward  chaining,  and 
leads  to  the  optimizations  described  in  Section  3. 


2.2  Formal  Semantics 

So  far  my  exposition  of  the  meanings  of  the  various  types  of  rule  has  been  informal.  But  below  I  will 
introduce  some  fairly  intricate  algorithms  for  making  inferences  using  those  rules,  and  we  will  want  to 
verify  that  the  algorithms  work.  Hence,  I  need  to  be  more  formal  about  the  semantics  of  the  rules. 


Definition  1  A  world  state  is  a  function  from  proposition  symbols  to  {#T.  #F.  J.}. 


A  world  state  s  can  be  considered  an  assignment  of  truth  values  to  every  proposition  symbol.  I  use  #T 
and  #F  to  denote  boolean  values;  X  means  “undefined,"  or  “inconsistent.”  The  state  F  assigns  #F  to 
every  proposition  symbol. 

I  will  extend  the  meaning  of  states  so  that  they  apply  to  boolean  combinations  of  propositions  in  the 
obvious  way.  In  this  paper,  propositions  will  be  notated  as  ground  atomic  formulas,  but  that  notation 
is  mainly  a  frill  All  we  require  is  a  supply  (possibly  infinite)  of  symbols  for  propositions,  which  I  will 
call  V.  Similarly,  we  need  a  supply  of  event  symbols,  which  I  will  call  Q. 


Definition  2  An  occurrence  is  a  pair  (e,t),  where  e  is  an  event  (type)  and  t  is  a  particular  time,  a 
nonnegative  real  number  The  event  of  the  occurrence  is  e  and  the  date  is  t.  VVe  will  write  such  an 
occurrence  as  e  1 1 


Definition  3  An  occurrence  sequence  is  a  sequence  of  occurrences,  ordered  by  date.  If  C  is  a  finite 
occurrence  sequence,  then  we  write  its  length  as  length(C).  If  the  dates  in  C  are  bounded,  we  write 
duration(C)  to  mean  the  least  upper  bound  of  those  dates 


Obviously,  if  C  is  finite,  duration(C)  =  date(Citn9t^C))- 


Definition  4  A  world  of  duration  L,  where  L  is  a  real  number  >  0,  is  a  complete  history  of  a  stretch  of 
time  of  duration  L.  That  is,  it  is  a  pair  (C,  H),  where  C  is  an  occurrence  sequence,  such  that  the  date 
of  the  last  occurrence  is  <  L,  and  H  is  a  function  from  [0,  L ]  to  world  states.  H( 0),  the  initial  state, 
must  be  F,  the  state  in  which  all  proposition  symbols  V  are  false.  If  <i  <  t2  and  H(t i)  ^  H(t2)  then 
there  must  be  an  occurrence  e  1 1  €  C  with  <  t  <  t2. 
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If  W  =  { C ,  H)  is  a  world  of  duration  L  and  A  is  a  proposition,  and  t  is  a  date  <  L ,  I  will  write  (A  1  t)(W), 
read  “ A  after  /  in  W”  to  mean  that  there  is  some  6  >  0  such  that  for  all  t',  t  <  t'  <  t  +  6,  H(t')(A). 
Similarly,  I  will  write  ( A  J  O(^).  read  “A  before  i  in  W to  mean  that  there  is  some  6  >  0  such  that 
for  all  t  —  6  <  t'  <  t,  H(t')(A).  If  c  is  an  occurrence,  I  will  write  A]  c  and  A 1  c  to  mean  A  f  date(c) 
and  A 1  date(c). 

It  should  be  obvious  that  in  any  world  W  =  (C,  H)  where  c  and  d  are  consecutive  occurrences  €  C,  and 
for  any  proposition  A,  (A  T  d)(W)  =  ( A 1  c)(W)  ,  and  for  all  t,  date[c)  <  t  <  date(d),  (4  1  t)(W')  =  (A  T 
t)(W)  =  {Aie){W). 

The  next  step  is  to  attempt  to  define  the  idea  of  a  model  of  a  probabilistic  theory. 


Definition  5  (Actually,  an  attempted  definition)  If  T  is  a  set  of  rules  as  described  in  Section  2.1,  C  is 
an  occurrence  sequence  of  length  n  (the  exogenous  occurrences),  and  L  is  a  real  number  >  duration(C), 
then  an  L-model  of  T  and  C  is  a  pair  where  W  is  a  set  of  worlds  of  duration  L  such  that  for 

each  ( C,H )  €  W,  C  C  C,  and  M  is  a  probability  measure  (Breiman  1969)  on  W  that  obeys  certain 
restrictions,  which  I  will  now  describe. 

We  cau  consider  A]  t,  A  [t,  and  e  [t  to  be  boolean  random  variables,  defined  on  the  “outcome  set” 
W.  As  usual,  we  can  combine  random  variables,  letting  M(->A)  =  1  -  M(A),  M(A\B)  =  ,  etc. 

Define  A,  the  annihilation  of  A,  where  A  is  a  conjunction,  to  be  the  conjunction  of  the  negations  of 
the  coqjuncts  of  A.  (Example:  P  A  ->Q  =  -<P  AQ  )  For  {W.M)  to  be  a  model,  the  measure  W  must  be 
constrained  to  fit  the  rules  T  as  follows: 


1.  Initial  blank  slate:  For  any  proposition  A  €  V,  M(A  TO)  =  0. 

r 

2.  Event-effect  rules  when  the  events  occur:  If  T  contains  a  rule  instance  Aje  |— *.  B  or  a  rule  instance 

r 

B  |  E\A,  then  for  every  date  t,  require  that,  for  all  nonempty  conjunctions  C  of  literals  from 
B 

M(Cit  \  E It  A  AU  /^B]t)  st  r 


3.  Static  backward-chaining  rules:  If  T  contains  a  rule  instance  B.  —  .A,  then 

M(B[t  |.41<)=  1 


4.  Event-effect  rules  when  the  events  don't  occur:  Suppose  B  is  an  atomic  formula,  and  let  R  =  {/?,} 

be  the  set  of  all  instances  of  |—  .  or  .  «—  |  rules  whose  consequents  contain  B  or  ~'B.  If  .ft,  = 
,  Vi  Pi 

Ai/Ei  |— < •.  Ci  or  C,  |  Ei\Ai,  then  let 

Di  =  .4,  A  C, 

Let  S  be  the  set  of  all  the  .  — .  rules  in  whose  consequents  B  occurs,  and  let 

.4=  V  (antecedent(r)) 

r€S 


i 


(The  Di  or  A  may  be  identically  falaa.  e.g.,  in  the  case  where  R  or  S  is  empty.)  Then 

M(Blt  |  BU  A  N)  =  1 

M(B[t  |  -’Bit/'  N)  =  0 

where 


N  =  (~’£'i  1 1  V  ->D i) 
A(— «£T2 1 1  V  ‘'Dj) 
A  .  .  . 

A-<i4 


5.  Event-occurrence  rules:  For  every  time  point  t  such  that  no  occurrence  with  date  t  ia  in  C,  and 
every  event  type  E,  let  R  be  the  set  of  all  rule  instances  of  the  form  -—I  £  in  T.  and  suppose 
that  S  is  an  arbitrary  nonempty  subset  of  R.  and  let  A  =  f\R  es^J'  anc*  *et  =  I/^;1 

d, 

where  Rj  =  A}  .  —  |  £.  Then  if  M(A  f  t)  ^  0,  require  that 

M(  some  occurrence  of  E  between  t  and  t  +  di  |  A  T  /)  =  A5  dt 
If  N  =  A R,^R~'^r  tIien 

Ai(aome  occurrence  of  £  between  t  and  t+d<|AM<)  =  0 
In  most  theories,  R  contains  zero  or  one  rule  instance.  In  the  former  case,  we  require 
M(  some  occurrence  of  £  between  t  and  t  +  dt)  =  0 
d 

In  the  latter  case,  with  just  one  instance  A  .  — |  e.  we  require 

M{  some  occurrence  of  £  between  t  and  /  +  dt  |  .4  1 1)  =  dt/d 

M(  some  occurrence  of  £  between  t  and  t  +  dt  \  ->-4  1 1)  =  0 


6.  Conditional  independence:  If  one  of  the  previous  clauses  defines  a  conditional  probability  M(a  |  0), 
which  mention  times  t,  then  a  is  conditionally  independent,  given  a,  of  all  other  random  variables 
mentioning  times  on  or  before  t.  That  is,  for  an  arbitrary  y  mentioning  times  on  or  before  t, 
M(a  |  0  A  7)  =  M{a  |  0). 


Clause  5  of  this  definition  is  stated  tersely  but  imprecisely.  To  make  it  rigorous,  we  must  replace  all 
statements  of  the  form  M(  some  occurrence  of  e  between  t  and  t  +  dt . . .)  =  x  dt  with 


iim 

Al  — 0 


M(  some  occurrence  of  e  between  t  and  t  +  A< . . .) 
— 


x 


This  quantity  has  the  character  of  a  probability  density,  and  it  is  well  defined  only  if  M  is  defined  over 
any  small  interval  around  almost  every  point  t.  There  is  a  time  point  associated  with  every  real  number, 
so,  if  a  theory  contains  rules,  each  of  its  models  must  have  an  uncountably  infinite  set  of  worlds. 
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In  what  follows,  I  *.'1I  refer  to  this  probability  density  with  the  slightly  misleading  notation  M(e  j 

d 

t . . .).  So,  even  r.  _»re  tersely,  we  can  state  the  constraint  on  a  theory  with  one  rule  A  .— *|  E  governing 
occurrences  ^  E  as: 

M(E[i\A\t)  =  1/d 
M{E[t\^A\t)  =  0 

In  fact,  I  already  made  use  of  this  notation  in  clauses  2  and  4  of  Definition  5.  The  meaning  of  M(-  |  c), 

where  c  =  E[t,  can  only  be  rigorously  specified  using  limits  if  T  includes  .-*|  rules. 

Because  of  this  infinity,  we  cannot  assume  that  Definition  5  makes  sense  without  putting  in  a  little  more 
work  to  show  that  there  is  exactly  one  M  that  satisfies  it.  The  work  is  what  you  would  expect:  a  limit 
process  that  defines  M  as  the  limit  of  a  series  of  discrete  approximations  The  details  are  important  but 
somewhat  tortuous,  so  I  have  put  them  in  an  appendix.  If  you  want  to  skip  the  appendix,  you  can  just 
assume  that  Definition  5  defines  a  unique  probability  distribution. 

Let  me  point  out  some  consequences  of  Definition  5  that  are  fairly  obvious  even  without  the  detailed 
analysis.  Please  attend  to  the  role  of  negation  in  the  clauses  of  Definition  5.  Suppose  we  have  a  rule 
e  |— *-  -<A.  Then  clause  2  says  that  A4(->A  [e  |  e  A  .4  |  e)  =  r.  Because  the  truth  values  of  propositions 
can  change  only  at  occurrences,  this  formula  means  that  any  occasion  of  A  that  persists  to  an  occurrence 
of  e  gets  clipped  with  probability  r. 

Clause  4,  with  its  negated  annihilations,  may  sound  daunting,  so  I  will  give  an  example.  Suppose  we 
have  three  rules 

0.8 

PAq/squirt  |— .  BAC 

0.6 

P/squirt  |  — .  -’BAD 

B  .  — .  E 


Then  clause  4  states  that 


M(B\t  |  B]tf\N)  =  1 
M(B\t  |  -,BU*N)  =  0 

where  N  is  defined  as 


(-’squirt  \  t  V  -4>  1 1  V  -4}  f  t  V  B  T  t  V  C 1 1) 
A  (^squirt  |  <  V -iP  I  <  V -iB  t  <  V  D  T  f) 
A  — <E  i  < 


One  consequence  of  these  rules  is  that  if  T  contains  two  rule  instances  that  specify  different  probabilities 
for  their  consequents  in  some  worlds  for  an  event  sequence  C,  then  there  is  no  model  of  T  and  C.  Such 
a  theory,  event-sequence  pair  is  said  to  be  inconsistent.  A  theory  is  consistent  if  it  is  consistent  for  all 
event  sequences.  For  example,  the  theory 

0.5 

START  A 

0.8 

A/bang  |— •_  BAC 

0.6 

A/bang  |— .  BAD 
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is  inconsistent  for  event  sequence  (START  1  O.b&ngl  1)  because  half  the  worlds  in  any  model  must  have 
A  i  START,  and  those  worlds  assign  two  different  probabilities  to  B  after  bang  For  event  sequences  not 
containing  bang,  the  theory  is  consistent.  Observe  the  impact  of  assigning  probability  to  every  subset  of 
the  consequent  of  a  rule.*  If  A  is  true  before  a  bang,  then  the  second  rule  states  that  B,  C,  and  BAC  must 
all  have  probability  0.8  afterward.  If  we  simply  assigned  a  probability  to  BAC,  then  the  theory  would 
be  consistent  after  all,  but  the  probabilities  of  the  individual  occasions  would  be  underdetermined.  For 
example,  we  could  assign  probabilities  0.6  to  BCD,  0.2  to  BCD,  and  0.2  to  BCD,  so  that  the  probability 
of  B  alone  was  1.  Other  assignments  would  give  B  different  probabilities.  My  semantics  avoids  such 
indeterminacies.® 


Compare  the  previous  theory  to  this  one: 


START 

0.5 

1  — - 

A 

A/bang 

0.8 

1  — - 

B 

'A/bang 

06 

I-- 

B 

This  theory  is  consistent  for  (START  10,  bang  1  1)).  because  only  one  of  the  B  rules  is  ever  firable. 

Another  way  for  inconsistency  to  develop  is  for  a  .  rule  to  disagree  with  a  |— -  rule.  However,  the 
following  theory  is  consistent: 

0.8 

bang  I—.  A 

0.6 

bang  | — .  ->A 

If  a  sequence  of  bangs  occurs,  then  there  are  worlds  where  both  rules  fire.  However,  the  first  rule  causes 
occasions  of  A  to  be  created,  and  the  second  causes  them  to  be  clipped.  Hence  the  two  rules  can  never 
fire  at  the  same  eventinstant. 

2.3  An  Abstract  Algorithm 

In  a  later  section,  I  will  discuss  the  actual  implementation  of  the  algorithm.  Here  I  will  describe  an 
idealized  version  that  makes  it  clear  how  the  algorithm  works.  One  of  the  idealizations  I  will  make  is 
to  ignore  rules,  which  clutter  the  exposition.  I  will  return  to  them  in  Section  3.  An  idealization 
I  will  not  make  is  to  ignore  . «—  |  rules.  If  we  got  rid  of  backward  chaining,  then  TL-RETRIEVE  would 
never  have  to  look  before  the  time  point  it  had  been  asked  about,  and  TIMELIIE-ADVAICE  would  do  all 
the  inferential  work.  However,  that  would  require  us  to  initialize  the  timeline  with  every  proposition 
the  system  believes.  As  I  will  discuss  in  greater  depth  in  Section  3.1,  that’s  an  awkward  requirement  to 
meet.  Hence  we  must  adopt  a  style  of  “lazy  projection”  in  which  points  along  the  timeline,  especially 
the  initial  point,  are  constructed  incrementally  as  queries  are  posed  about  them.  (Cf.  Hanks  1990a.) 

4  An  alternative  way  to  state  the  condition  is  to  take  the  consequent  of  a  rule,  and  generate  all  possible  combinations 
of  “signs”  for  the  atomic  formulas  in  it.  Every  combination  except  the  one  given  must  have  probability  zero.  For  the  case 
at  hand,  the  second  rule  would  assign  conditional  probability  zero  to  B  A  -.C,  ->B  A  C,  and  -iB  A  ->C. 

5Of  course,  by  making  additional  assumptions,  it  is  possible  to  assign  probabilities  in  cases  like  these.  One  such 
approach  is  explored  by  Thiebaux  and  Hertzberg  (1992).  In  my  experience,  it  is  usually  preferable  to  rewrite  rules  to  avoid 
inconsistencies. 
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A  timeline  keeps  track  of  events  and  the  occasions  that  they  delimit.  The  events  are  totally  ordered,  and 
no  proposition  changes  in  truth  value  between  two  adjacent  events.  Hence  we  can  implement  timelines, 
essentially,  as  lists  of  eventinsiants,  defined  thus.  An  eventinstant  is  a  record  that  contains: 


•  A  date:  The  time  of  occurrence  of  the  eventinstant,  relative  to  the  beginning  of  the  timeline. 

•  Happenings:  The  events  that  occur  here  (zero  or  one). 

•  Overlapping  occasions:  Occasions  that  are  true  here  (that  is,  they  overlap  this  eventinstant,  either 
because  they  begin  here,  or  because  they  begin  before  and  are  not  clipped  here  or  before). 

•  Clipped  ocasions:  A  list  of  occasions  that  were  clipped  at  this  eventinstant. 

•  Established  queries:  A  list  of  atomic  queries  that  have  been  tested  at  this  eventinstant. 

•  Clipnotes:  A  table  of  (  atomic-formula,  boolean  )  pairs,  where  the  boolean  records  whether  the 
decision  was  to  clip  or  not  to  clip  the  atomic  formula  at  this  eveninstant 


The  actual  structure  of  an  eventinstant  is  more  complex,  but  this  will  do  for  now. 
Each  occasion  is  a  record  that  contains: 


•  A  proposition:  a  ground  atomic  formula 

•  Begin:  The  eventinstant  where  the  occasion  begins  to  be  true 

•  End:  The  eventinstant  where  the  occasion  ceases  to  be  true,  or  #F  if  that  eventinstant  is  not  yet 
known. 


The  fact  that  we  store  a  list  of  occasions,  each  associated  with  an  atomic  formula,  means  that  we  are 
making  the  closed-world,  assumption  for  occasions:  if  no  occasion  for  proposition  A  overlaps  a  time 
instant,  then  the  algorithm  will  take  A  to  be  false  at  that  time  instant. 

The  established-queries  list  and  the  clipnotes  table  of  an  eventinstant  play  a  crucial  role.  Once  a  query  has 
been  processed  at  an  eventinstant,  the  inferred  instances  must  be  stored  in  this  list.  Such  a  query  is  said 
to  have  been  established  at  this  eventinstant.  There  are  two  good  reasons  for  saving  this  information. 
First,  it  serves  as  a  cache.  When  backward-chaining  for  answers  to  a  query,  the  system  must  check 
previous  eventinstants  to  see  if  the  query  can  be  infered  there  and  shown  to  persist  to  the  point  of 
interest.  If  a  query  is  repeated  at  every  time  point,  which  is  a  common  occurrence  in  our  application, 
this  check  will  propagate  back  to  previous  eventinstants  many  times.  The  cache  saves  repeating  rule 
application.  Second,  and  even  more  important,  we  simply  cannot  run  a  probabilistic  rule  more  than 
once  at  a  given  eventinstant.  Once  the  answer  to  a  query  has  been  randomly  selected,  we  must  record 
the  outcome,  and  avoid  doing  another  random  selection  later. 

A  timeline  is  then  essentially  a  list  of  eventinstants.  In  the  Lisp  implementation,  the  list  is  kept  in 
reverse  chronological  order,  so  that  adding  to  the  timeline  requires  COISing  a  new  eventinstant  on,  and 
backward  chaining  requires  CDAing  through  the  list. 
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Query  B? 


not  A  <-|  E2 


B  <-|  E3\A 


El 


E2 


A  t  C  _<-t  ElAD.- . . 

h  ■ —  Create  A 


E3 

0 

0 

Subquery  AT*-*' 
bring  forward  *  ...? 


*  E4 

not  C  <-|  E4 


Create  C 


bring  forward* 


. .  .? 


Figure  1:  Interleaving  Backward  Chaining  and  Occasion  Creation 


I  will  use  the  term  time  point  to  refer  to  a  position  in  this  list.  The  initial  time  point  corresponds  to  the 
beginning  of  the  list.  Every  noninitial  time  point  is  associated  with  an  eventinstant,  namely,  the  one  that 
just  occurred  (the  CAR  of  the  list  in  the  implementation).  Because  of  this  near  one-to-one  correspondence 
between  time  points  and  eventinstants,  I  will  refer  to  the  date,  happenings,  etc.  of  a  noninitial  time 
point,  meaning  the  date,  happenings,  etc.  of  the  associated  eventinstant.  The  past  of  a  noninitial  time 
point  is  the  time  point  that  is  its  immediate  predecessor  (its  CDR  in  the  implementation).  Because  no 
occasion  changes  in  truth  value  between  eventinstants,  we  do  not  really  have  continuous  time,  but  in 
reasoning  about  endogenous  events  the  system  will  have  to  choose  real-valued  dates  at  which  to  place 
the  next  event.  In  describing  this  part  of  the  system,  I  will  use  the  term  time  instant  to  describe  one  of 
the  uncountable  number  of  anonymous  points  between  two  eventinstants. 

TL-RETRIEVE  is  the  basic  entry  point  to  the  temporal-inference  system,  so  I  will  start  by  describing  it. 
It  takes  as  argument  a  query,  a  time  point,  and  a  timeline.  It  returns  two  values:  a  list  of  substitutions 
that  answer  the  query  (possibly  empty),  and  a  list  of  new  occasions  created  while  answering  it  (usually 
empty).  This  second  value  is  quite  important.  Even  though  a  query  may  have  been  asked  at  a  point 
late  in  a  timeline,  answering  it  may  require  running  a  . «—  |  rule  at  a  point  arbitrarily  early  in  the 
timeline.  An  important  special  case  is  rules  that  trigger  off  START.6  When  a  rule  fires,  it  creates  new 
occasions,  which  persist  for  a  while,  but  may  get  clipped  eventually.  Getting  TL-RETRIEVE  right  requires 
synchronizing  this  activity  with  backward  chaining.  When  new  occasions  are  created,  the  system  will 
be  in  the  midst  of  exploring  answers  to  a  query  at  a  future  time  point.  See  Figure  1.  It  will  be  several 
layers  down  in  a  recursion  involving  possibly  a  cascade  of  j  rules.  If  the  system  tried  to  figure  out  the 
lifetime  of  an  occasion  as  soon  as  it  was  created,  it  would  end  up  starting  several  new  backward-chaining 
computations  in  the  same  place  in  the  timeline  as  the  existing  computation,  and  inconsistent  answers 
and  infinite  recursion  would  ensue.  On  the  other  hand,  if  it  postponed  figuring  out  the  lifetime  until  the 
dust  had  settled  on  the  current  computation,  then  the  current  computation  might  not  see  the  correct 
lifetime  when  it  resumed  at  later  time  points. 

The  only  solution  is  to  interleave  backward  chaining  with  creation  of  new  occasions.  When  TL-RETRIEVE 
returns,  it  guarantees  that  the  timeline  is  in  a  consistent  state  up  through  the  given  time  point,  and 
that  the  new  occasions  it  is  returning  persist  that  far.  If  it  was  called  recursively,  its  caller  must  check 
whether  the  occasions  persist  to  the  next  time  point.  If  they  do,  they  get  returned  up  the  line  to  the  next 

®and,  in  the  actual  implementation,  timeline  initializers,  described  in  Section  3. 
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caller;  otherwise,  they  get  clipped.  This  persistence  check  is  calling  “bringing  the  occasions  forward.” 

I  will  present  algorithms  in  a  pseudo-code  style  with  vaguely  Algol-like  syntax.  Italics  will  be  used  for 
comments,  and  for  parts  of  the  code  that  are  not  worth  going  into  detail  about.  I  will  notate  multiple- 
value  return  with  angle  brackets.  If  a  function  F  returns  an  expression  like  <x\ ,  *2 »  •  -  •  >■  then  its 
caller  can  capture  those  values  by  saying 

Let  <vi ,  V2,  . .  >  =  F(. . . ) 

— body-of-Ltt — 

As  this  example  shows,  scope  is  indicated  with  indentation  wherever  possible.  To  indicate  random  binary 
choices,  I  write  a  call  to  a  procedure  random-choice(pro6). 

Here  is  the  code: 

TL-RETRIEVE (query , t imepo int , t imeline ) 

If  query  is  a  conjunction, 

then  call  TL-RETRIEVE  on  each  conjunct  and  combine  the  results, 
ordering  the  conjuncts  so  ”  conjuncts  occur  last. 

Else  if  t  imepo  int  is  the  initial  timepoini  of  timeline 
then  <if  query=true  or  for  some  A 
then  ( empty  substitution) 
else  ()t 

()> 


Else  if  query  is  of  form  -<Q, 

then  Let  <  anses,  nesoccs  >  =  TL-RETRIEVE(Q,timepoint, timeline) 

<if  anses  =  ()  then  (.empty  substitution) 
else  (), 

neeoccs> 

else  Let  nevoccs  =  Maybe-establish-query(query .timepoint, timeline) 
<Match-against-occasions(query .timepoint) , 
nevoccs> 

Hatch-against-occasions (query .timepoint ) 

Match  query  against  all  occasions  in  overlapping-occasions(timepoint) 
Return  a  list  of  substitutions,  one  for  each  successful  match 

Naybe-establish-query (query , timepoint , t imel ine) 
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If  query  is  a  variant  of  some  member  of  established-queries  (tinepoint) 
then  () 

else  Establish-query(query .tinepoint , tineline) 

Establish-query (query , t inepoint , t incline ) 

If  tinepoint  is  initial  time  point  of  timeline 
then  () 

else  Let  fron-past  = 

Br ing-f ore  ard(Naybe-establish-query (query .past (tinepoint) , tineline) . 
t inepo int , t incline ) 

fron-past  U  Eatablish-af ter (query .tinepoint, tineline) 

Es tabl ish-af t  er ( query , t inepo int, timeline) 

Let  newoccs  =  ’() 

P 

For  each  rule  Q\  A  <J2  A...  A  Q„  .<— |  e\C , 

where  query  unifies  with  a  Q,  and  event  of  tinepoint  unifies  with  e 
with  unifier  $ 

newoccs  :  =  newoccs  U  Try-back-chain-rule(rt»/e, 9, tinepoint, tineline) ; 
add  query  to  established-queries(timepoint) ; 
newoccs 

Try-back-chain-rule (rule , substitution , t inepo int , t imel ine) 

P 

;rule  is  of  form  Q  i  A  Q2  A  ...  A  Q„  |  e\C 
Let  precond  =  Apply  substitution  to  C 
Let  <anses,occs>  =  TL-RETRIEVE (precond  A  ->Qi  A  ...  A  ->Qn$ 

past (tinepoint) .timeline) 

Let  newoccs  =  Bring-forward(occs, tinepoint, tineline) 

For  each  p  in  anses 
newoccs  :=  newoccs 

U  Conclude-from-rule(rule,^, tinepoint, tineline) ; 

Note  that  query  is  established  at  tinepoint; 
newoccs 

Conclude-f r on-rule (rule , subatitut ion , t inepo int , t incline ) 

p 

;rule  is  of  form  Qi  A  Qj  A  ...  A  Qn  |  e\C 
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p 

;  or  C/t  |— ►.  Qi  A  Qi  A...  A  Q„ 

L«t  £)j  A  D?  A  . . .  A  Dn  -  Apply  substitution  to  Q  i  A  Q 2  A  ...  A  Qn 
and  prob  3  Apply  substitution  to  p 
and  nanoccs  =  ’() 

If  random-choice (prob)  ;  (rule  fires) 
then  For  each  Di 

If  of  form  -<A 

then  if  no  entry  (A,...  )  £  clipnotes(tinepoint) 

;  No  previous  decision  has  been  made  whether 

;A  is  clipped  at  tinepoint 

then  add  (.4,  #T)  to  clipnotes(timepoint) 

Else  if  Di  £  established-queries(timepoint) 
then  ;  Note  that  Di  is  established  at  timepoint 

add  Di  to  established-queries (t imepoint ) ; 

Let  occ  =  Create  new  occasion  with  proposition  =Di 

and  begin-timepoint 
and  end=#F 

add  occ  to  overlapping-occasions (t imepoint ) ; 
neuoccs  :*  cons (occ .newoccs) 

else  For  each  Di 

If  of  form  -iA 

then  if  no  entry  (A,...  )£  cl ipnotes( timepoint) 

;No  previous  decision  has  been  made  whether 
;A  is  clipped  at  timepoint 
then  add  (j4,  #F)  to  clipnotes (timepoint) 
else  ;Note  that  Di  has  been  established  at  timepoint 
add  Di  to  established-queries (timepoint) ; 


nesoccs 


Notice  how  the  established-queries  and  clipnotes  tables  are  used  by  these  routines.  The  basic 
idea  is  that,  for  a  given  time  point,  no  query  is  processed  more  than  once,  and  no  decision  to  create 
or  clip  an  occasion  is  made  more  than  once.  Every  time  one  of  these  things  happens,  it  is  recorded  in 
the  appropriate  table  to  prevent  its  happening  again.  It  would  be  possible  to  suppress  the  established- 
queries  manipulations  in  Haybei-establish-query  and  Establish-aftor,  although  it  would  be  less 
efficient.  It  is  not  possible  to  avoid  the  establish- queries  manipulations  in  Concluds-f  rom-rule,  because 
consideration  of  the  same  literals  can  arise  from  consideration  of  different  queries.  For  example,  the  rule 
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u.g 

B(x,  y)  .<— |  E\A(x,  y)  can  lead  to  consideration  of  the  literal  B(a,  b)  from  both  the  query  B(a,  y)  and  the 
query  B(x,b).7 

Unlike  the  established-queries  table,  the  clipnotes  table  records  the  outcome  of  the  clipping  decision  as 
well  as  the  fact  that  it  was  made.  As  we  will  see  when  considering  |— ►.  rules,  when  the  system  notices 
whether  a  proposition  is  clipped  at  a  time  point,  it  may  or  may  not  know  whether  an  occasion  with  that 
proposition  overlaps  that  time  point.  So  the  algorithm  must  save  the  clipping-decision  result  for  future 
reference.  Bring-f orward  then  retrieves  the  outcome  from  the  table  to  see  what  is  clipped.  Here’s  the 
code: 

;  This  procedure  is  responsible  for  bringing  new  occasions  forward 
;and  checking  whether  they  survive  through  tinapoint 
Bring-f orward ( oc  cas ions , t imepo int , t inel ina ) 

Lat  to-futura  =  ()  ;  The  list  of  survivors  we’re  going  to  asemble 
For  aach  occ  €  occasions 

Lat  <clipped,newoccs>  *  Been-clipped(occ,timepoint .timeline) 
to-futura  :=  to-futura  U  nanoccs; 

If  clippad 

than  and(occ)  :«  tinapoint 

also  (Add  occ  to  ovarlapping-occas ions (tinepo int ) ; 
to-futura  :=  consfocc, to-futura)) ; 

to-futura 

;  Test  for  whether  occasion  gets  clipped  at  tinapoint 
•.Return  two  values:  boolean  indicating  whether  it  got  clipped, 

;plus  (subtle  requirement)  new  occasions  created  when  deciding 
;  whether  this  one  is  clipped(!) 

Baan-clippad  ( occasion , t inepoint , t inel ina ) 

Lat  prop  -  proposition(occasion) 
and  nawoccs  =  ’() 

P 

For  aach  rula  of  tha  forn  Q\  A  Q 2  A  ...  A  Q„  |  e\C 
where  some  Q,  is  of  form  -> P, 
and  prop  unifies  with  P 
and  avant  of  tinapoint  unifies  with  e 
with  unifier  9 

nawoccs  :  =  nawoccs  U  Try-back-chain-rule(rula, 9, tinapoint , tineline) ; 

7  An  alternative  idea  would  be  to  keep  a  “used  rule  instance"  table  in  addition  to  the  established-queries  table. 
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<if  Then  is  an  entry  (prop.c)  in  clipnotas(tiaapoint), 
than  c 
•ls«  #F, 
nawoccs> 


The  following  lemma  is  obvious  but  useful: 

Lemma  1  Concluda-froa-rule  makes  a  decision  about  creating  or  clipping  an  occasion  at  a  time 
point  only  if  the  decision  has  not  been  made  before. 


Proof:  Each  such  decision  is  recorded  in  the  astablished-querias  or  clipnotas  table  for  the  time 
point,  and  blocks  repetitions.  QED 

To  prove  that  TL-RETRIEVE  works,  I  first  state  two  definitions. 


Definition  6  A  timeline  is  properly  established  through  time  point  t  if  and  only  if  t  is  initial  or  for 
every  time  point  p  before  or  coinciding  with  t,  and  every  occasion  n  =  (A,  Ci ,  C2)  in  any  eventinstant’s 
overlapping-occasions  list,  n  €  overlapping-occasions (p)  if  and  only  if  date(c  1)  <  date(p)  < 
date(c2),  taking  any  date  to  be  <  date(c2)  when  date(c2)  —  #F. 


Definition  7  A  timeline  is  properly  clipped  through  time  point  t  if  for  every  time  point  p  before  or  coin¬ 
ciding  with  t,  except  the  initial  one,  if  c  €  overlapping-occasions(past(p)),  where  c  has  proposition 
A,  then 

•  there  is  an  entry  (A,  #T)  €  clipnotas(p)  if  and  only  if  c  0  ovarlapping-occasions(p); 

•  if  there  is  no  entry  for  A  in  clipnotes(p)  then  there  is  no  |— ► .  or  _ «—  |  rule  whose  consequent 
includes  a  literal  ->A',  where  A'  unifies  with  A. 


Now  to  state  that  TL-RETRIEVE  works  correctly: 


Lemma  2  If  a  timeline  L  for  a  consistent  theory  T  is  properly  established  and  properly  clipped  through 
time  point  t,  then  TL-RETRIEVE(?,t,  L)  will 

1.  make  a  decision  about  creating  or  clipping  an  occasion  at  a  time  point  only  if  the  decision  has  not 
been  made  before; 

2.  make  and  record  decisions  about  occasions  whose  propositions  are  subsumed  by  atomic  formulas 
occuring  in  q,  such  that  the  truth  value  of  all  instances  of  q  at  t  will  be  decided  and  recorded; 

3.  make  all  decisions  based  on  the  correct  conditional  probabilities,  as  laid  out  in  Definition  5; 

4.  leave  L  properly  established  and  properly  clipped  through  <; 
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5.  return  all  answers  that  follow  from  the  decisions  made,  plus  a  list  of  all  occasions  created  that 
overlap  t,  each  of  which  will  be  of  the  form  ( A,c ,  # F). 


Proof:  The  first  clause  follows  from  the  fact  that  Concluda-f ron-rula  is  the  only  routine  that  creates 
occasions,  and  Bring-forvard  is  the  only  one  that  clips  them.  Both  respect  the  decisions  taken  by 
Concluds-i roa-ruls ,  so  clause  1  follows  by  Lemma  i. 

Clause  2  will  follow  if  we  can  show  that  it  holds  for  any  atomic  query,  because  TL-RETRIEVE  breaks  a 
compound  query  down  into  atomic  pieces. 

Clauses  2-5  are  proven  by  induction  on  the  number  of  eventinstants  up  through  t,  taking  the  query  to 
be  atomic  as  just  mentioned.  If  t  is  the  initial  time  point  of  L,  then  L  will  vacuously  remain  properly 
clipped  and  established  through  t.  No  probabilistic  decisions  will  be  made  or  recorded,  and  only  formulas 
of  the  form  trus  or  ->A  will  be  taken  as  true.  No  occasions  will  be  created  or  returned,  and  the  empty 
substitution  or  nothing  will  be  returned. 

Now  assume  clauses  2-5  of  the  theorem  are  true  up  to  t,  and  consider  t's  successor  t '  in  L. 

Suppose  that  the  algorithm  is  about  to  make  a  decision,  in  Concluda-froa-rula,  about  whether  to  clip 
or  create  an  occasion  n  with  proposition  .4.  Conclude-from-rula  is  called  in  Try-back-chain-ruls, 
after  a  call  to  TL-RETRIEVE  with 

query  =  precondition  of  rule  A  consequent  of  rule 

and  tin*  point  t  —  <  past  (current  time  point).  This  query  must  have  succeeded,  thus  finding  a  Arable 
rule  instance  whose  consequent  contains  A  or  -cA.  (Because  T  is  consistent,  there  will  be  just  one  such 
rule  instance.)  By  the  induction  hypothesis,  the  recursive  call  to  TL-RETRIEVE  will  make  enough  decisions 
about  this  subquery  to  fix  the  truth  values  of  all  its  instances  at  t ,  and  will  return  the  list  P  of  all  newly 
created  occasions  that  persist  through  t.  In  particular,  to  get  to  this  point  in  Concluds-froa-rula,  it 
must  have  found  the  substitution  corresponding  to  A.  Hence  when  it  makes  a  random  choice,  it  uses 
the  probability  from  the  rule  instance  obtained  using  this  substitution,  so  that  the  chance  of  creating  or 
clipping  A  is  as  described  in  clause  2  of  Definition  5. 

Now  consider  an  occasion  of  A  in  ov«rlapping-occasions(<),  such  that  no  rule  involving  A  or  -»A  ever 
becomes  Arable  at  t'  as  a  result  of  decisions  made  by  recursive  calls  to  TL-RETRIEVE.  Because  L  was 
properly  established  and  clipped  through  t'  when  we  started,  and,  by  induction,  it  is  properly  clipped 
through  t  after  all  recursive  calls  to  TL-RETRIEVE,  then  either  n  €ov«rlapping-occasions(t')  or  n  € 
some  new-occasions  list  returned  by  a  recursive  call  to  TL-RETRIEVE.  In  the  latter  case,  Bring-forvard 
will  have  been  called  on  the  new-occasions  list,  and,  by  the  induction  hypothesis,  decisions  will  have 
been  made  that  fix  at  false  the  truth  value  of  the  formula  corresponding  to  D  in  clause  4  of  Definition 
5.  Hence  Bring-f  orvard  will  add  n  to  the  overlapping-occasions  list  of  t' . 

To  prove  clause  5,  notice  that  the  occasions  returned  by  TL-RETRIEVE  come  from  two  sources:  those 
created  in  Conclude-f roa-rule  and  those  returned  by  recursive  calls  to  TL-RETRIEVE.  The  former 
category  all  get  returned,  and  all  have  end=#F.  The  latter  category  also  have  snd=#F  (by  induction 
hypothesis),  but  they  get  passed  through  Bring-f  orvard,  which  either  clips  them  and  resets  their  ands, 
or  returns  them,  still  with  «nd=#F. 
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Finally,  we  need  to  show  that  the  timeline  remains  properly  established  and  clipped  through  t'.  Let  n 
be  an  occasion  €overlapping-occasions(f).  If  n  was  present  before  this  call  to  TL-RETRIEVE,  then 
its  status  in  overlapping-occasions^  and  clipnotes(t')  will  not  be  changed.  So  suppose  that  n  is 
a  newly  created  occasion  of  A.  Because  it  is  new,  and  doesn’t  start  at  it  will  have  been  returned 
by  some  recursive  call  to  TL-RETRIEVE.  It  will  then  be  checked  by  Been-clipped,  and  will  be  omitted 
from  overlapping-occasions(t')  if  only  if  an  entry  (A,  # T)  is  added  to  the  clipnotes  for  t' .  Hence  the 
timeline  will  remain  properly  clipped.  To  show  it  remains  properly  established,  we  also  need  to  consider 
occasions  that  begin  at  t'.  But  these  are  always  put  in  overlapping-occasions^')  QED 

The  other  main  entry  point  to  the  system  is  TIHELHE-ADVAICE,  which  adds  a  single  new  eventinstant 

to  a  timeline.  The  easy  part  of  its  job  is  to  run  |— *_  rules  for  the  new  eventinstant.  The  tricky  part 

d 

is  to  run  _  — »|  rules.  Recall  that  the  meaning  of  A  _-*|  e  is  that  over  any  tiny  interval  (duration 
“dt”)  where  A  is  true,  events  of  type  e  tend  to  occur  with  probability  dt/d.  What  this  entails  is  that, 
over  an  interval  of  length  /,  the  probability  that  an  e  does  not  occur  is  exp(— A/),  where  A  =  l/d 
(Breiman  1969, Feller  1970). 8  In  simple  cases,  such  a  rule  should  generate  a  random  series  of  e  events, 
according  to  a  Poisson  distribution.  But  every  time  an  event  occurs,  |— »  _  rules  can  change  the  set  of 
true  propositions.  Consider  these  rules: 


1 

E  |—  -  -'A 

the  first  rule  suggests  that  any  occasion  of  A  is  punctuated  by  random  occurrences  of  E  every  10  time 
units  (on  the  average).  But  the  second  makes  it  clear  that  in  fact  the  first  such  occurrence  will  be  the 
last. 

The  best  way  to  think  of  the  meaning  of  .— »|  rules  is  this:  At  any  time  instant,  there  is  a  set  of  firable 
-—♦I  rules.  Each  is  like  a  ticking  time  bomb,  with  random  elapsed  times  until  they  fire  or  until  the  next 
exogenous  event  occurs.  If  one  of  the  rules  fires,  a  new  event  is  created,  the  set  of  firable  rule  instances 
gets  recomputed,  and  the  process  resumes.  In  a  sense,  this  process  never  stops,  but  the  system  does  not 
bother  to  model  past  the  last  exogenous  event.  Hence  the  job  of  TIMELI1E-ADVAICE  is  to  tack  on  a  new 
exogenous  eventinstant,  then  fill  in  with  endogenous  events  the  time  interval  between  the  eventinstant’s 
predecessor  and  the  new  eventinstant.  I  will  call  this  the  time-passage  interval. 

It  is  a  well  known  fact  (see,  e.g.,  Breiman  1969)  that  a  set  of  independent  processes,  each  of  which 
behaves  as  described  by  a  < <\  rule  instance  with  frequency  A*,  can  be  modeled  as  a  single  process  with 
A  =  Aj.  Hence  the  chance  that  no  endogenous  event  occurs  in  a  time-passage  interval  of  length  /  is 
exP ^  an  event  does  occur,  then  the  chance  that  it  is  of  the  type  belonging  to  process  »  is 
A./A. 

I  will  now  present  the  algorithm.  I  use  the  notation  date(L)  to  denote  the  date  of  the  last  eventinstant 
in  timeline  L. 


\Add  a  new  eventinstant  at  the  end  of  tineline,  after  elapsed 
;time  units.  Put  an  event  of  type  event  at  the  new  eventinstant. 

*The  only  reason  to  use  d  instead  of  A  in  the  rule  is  that  in  the  implementation,  it  is  occasionally  useful  to  set  d  =  0. 
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TIMELI1E-AC” « ICE  ( event , elaps  ed , t imel in* ) 

Let  ad v data  3  date (timeline)  +  a lap* ad 
Pass-ti*e(timeline .advdate) ; 

Add- a vent inst  ant (advdat  a , t  imel ina ) ; 

Add-avant (avent .timeline) 

;  This  routine  fills  in  a  time-passage  interval  with  endogenous  events 
Pass-time(timeline .nartdata) 

Lat  rl  =  Event-rule-instances(timeline) 

If  not  empty (rl) 

than  Lat  A  =  }T(Ai)  for  avary  rule  instance  in  rl 
Lat  prob  =  i_e-M*wtdate  —  data(  timeline )) 

if  random-choice (prob) 

than  Lat  x  =  random  real  number  between  0  and  1 
Let  when  =  ~ '°«<‘-»xProb) 

Lat  which  =  pick  rule  t  from  rl 
with  probability  ^ 

One- endogenous -event (which .timeline , 

date(timeline)+when, 

nextdata) 


Event-rule- instances (timeline) 

Lat  instances3 () 

d 

For  avary  rule  A  _— >|  E 

Lat  <anses,newocc8>  =  TL-RETRIEVEM.last-time-point(timeline) .timeline) 
Ignore  nesoccs  —  they’ll  be  picked  up  by  the  call  to  Check-clipping; 

For  a  in  ansas 

add  instance  of  rule  for  a  to  instances ; 
instances 

;Add  event  from  rule-instance  to  timeline  at  time  nevdate, 

;  then  pass  time  until  thendate 

One-endogenous- event (rule- instanc e , t imel ine , nesdat  e . t  hendat  e ) 

Add-event instant (nevdate .timeline) ; 

Add-event ( event  from  rule-instance, timeline) ; 

Pas  s- 1 ime ( t imel ine , thendat  e ) 
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Add-wvwnt instant (date , t imwlins ) 

Add  one  more  eventinstant,  with  date  data,  to  the  list  in  tiaalina 

Add-s vwnt ( sv ant , tiaalina ) 

Add  avant  to  happaningaClast  evantinstant  of  tiaalina); 

Forw&rd-ch&infevent. timeline) ; 

Check- cl ipp ing ( t iaal ina ) 

Forward-chain (event .tiaalina) 

p 

For  every  rula  A/e  |— ♦.  B,  where  e  unifies  with  event  with  substitution  9 
Let  <anses,newoccs> 

3  TL-RETRIEVE(0A,ponultimate-time-point(timeline)  .tiaalina) 

Ignore  nawoccs  (see  comment  in  Event-rule-instances,); 

For  a  in  anses 

Concluda-f roa-rule(rule, a, last-time-point (tiaalina) .tiaalina) 

Chack-cl ipping ( t iaal ina ) 

For  each  occ  G  ovarlapping-occasions(penultimate-tima-point(tiaalina) ) 

If  Baan-clipped(occ,last-tiaa-point(tinalina) , tiaalina) 
than  end(occ)  : =  tiaepoint 

else  add  occ  to  overlapping-occasions(last-time-point(ti*eline) ) 

This  algorithm  is  somewhat  unusual  in  that  its  termination  condition  is  probabilistic.  It  can  in  principle 
loop,  interpolating  new  events,  forever,  as  Pass-time  calls  Ona-endoganous-avant  and  Ona-andoganous-avant 
calls  Pass-time.  Of  course,  the  probability  of  this  happening  is  zero. 

Lemma  3  If  a  timeline  L  for  a  consistent  theory  T  is  properly  ended,  and  properly  established  and 
properly  clipped  through  its  last  time  point,  then  TIMELISE-ADVAICE(e,  /,  L)  will 

1.  make  a  decision  about  creating  or  clipping  an  occasion  at  a  time  point  only  if  the  decision  has  not 
been  made  before; 

2.  make  all  decisions  based  on  the  correct  conditional  probabilities,  as  laid  out  in  Definition  5; 

3.  leave  L  properly  established  and  properly  clipped  through  t. 


21 


A 


C  G  E? 

- 1 - 1 - 1— 

/  \  ' 

Lait  exogenous  Lest  event 

event  before  I 


— I 

naxtdat* 


Possibly  identical 


Figure  2:  Generating  Autonomous  Event  E  [  t  Given  A  1 1 

Proof:  TIMELIIE-ADVAICE  adds  eventinstants  only  to  the  end  of  a  timeline,  using  Add-event  ins  t  amt. 
Add-event  then  runs  forward  chaining  rules,  calling  TL-RETRIEVE  on  the  past  of  the  new  timeline  (which 
=  the  last  time  point  of  the  old  timeline).  By  Lemma  2,  TL-RETRIEVE  leaves  the  timeline  in  the  proper 
state  up  through  that  time  point.  So  all  I  have  to  do  is  show  that  the  later  machinations  of  Add-event 
obey  clauses  1  through  3  of  the  lemma.  Clause  1  follows  because  Add-event  uses  Conclude-froa-mle 
(Lemma  1).  The  proof  of  clause  2  is  essentially  the  same  as  for  clause  3  of  Lemma  2,  again  because  of  the 
use  of  Conclude-fro«-rule.  The  proof  of  clause  4  is  obtained  by  inspection  of  Check-clipping,  which 
is  essentially  the  same  as  Bring-foreard,  but  applies  to  every  element  of  the  overlapping-occasions 
of  the  previous  time  point.  QED 

Lemma  4  TIMELIIE-ADVAICE  creates  new  events  in  accordance  with  clause  5  of  Definition  5. 


Proof:  For  simplicity,  we  will  consider  the  case  where  an  event  is  created  by  exactly  one  firable  rule 

d 

instance,  of  the  form  A  >|  E,  and  let  Ai  =  1/d.  What  we  want  to  prove  is  that 

P(E  occurs  in  interval  [t,f  +  dt)  |  A  |<)  =  A jdt 

in  accordance  with  clause  5  of  Definition  5.  In  this  formula,  P  stands  for  probability  defined  over 
execution  traces  of  TIMELIIE-ADVAICE  for  a  given  set  of  exogenous  occurrences.  There  are  an  infinite 
number  of  such  traces,  and,  if  we  wanted  to  be  technical  about  it,  we  would  define  carefully  how  to 
impose  a  measure  on  them.  However,  these  technicalities  would  take  us  far  from  the  topic  of  the  paper, 
and  would  not  reveal  any  unsurprising  phenomena,  so  we  will  just  use  common  sense. 

The  only  way  for  TL-RETRIEVE  to  conclude  that  A  is  true  at  t  is  for  it  to  conclude  that  A  is  true  after 
some  previous  occurrence  G,  and  for  no  other  events  to  be  added  to  the  timeline  from  then  until  t.  G 
must  follow  or  be  identical  to  C,  the  last  exogenous  event  before  t.9  See  Figure  2. 

Hence,  abbreviating  “an  occurrence  of  E  is  placed  in  interval  [f,t  +  dt)”  as  “E  [t”\  and  “no  event  is 
created  between  ti  and  <2  as  N(ti,t2)”\  and  “the  set  of  firable  rule  instances  at  t\  is  F”  as  FIG, 

9There  are  execution  traces  in  which  an  infinite  number  of  events  happen  after  A  becomes  true  (in  fact,  there  are  some 
in  which  A  changes  truth  value  infinitely  often  just  before  t),  but  the  set  of  all  such  execution  traces  is  of  measure  0,  and 
TL-KSTIIEVS  will  run  forever  in  such  cases,  never  concluding  that  A  is  true  at  t. 
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we  have 


P(E{t  |  A|<)  =  P{E[t  \  3G,  F(date(C)  <  date(G)  <  t  A  A[G  A  F{G  A  N(date(G),t))) 


Consider  an  arbitrary  occurrence  G  with  date(C)  <  date(G)  <  t  <  naxtdata,  and  consider  an  arbitrary 
timeline  in  which  G  has  just  been  added,  and  Event-rule-instances  has  then  picked  the  set  F  of  all 
firable  rule  instances  (which  it  will  by  Lemma  2).  Their  frequencies  A,  add  up  to  A.  (Ai  is  one  of  them,  of 
course.)  Let  At  =  nextd&te  -  date(G),  the  size  of  the  interval  into  which  events  may  or  not  be  placed. 
If  the  algorithm  creates  an  event,  it  creates  it  with  date  vhen+dafe(G),  and  the  cumulative  distribution 
of  shen  has  P(w hen  +  date(G)  <t)  = 

1  _  e-A(«-So«*(G)> 

1  — e-AA< 

This  formula  can  be  verified  by  finding  the  cumulative  distribution  of  vhan  given  that  x  is  uniformly 
distributed  (the  standard  “inversion  trick”  for  generating  nonuniform  random  numbers;  see,  e.g.,  Bratley 
et  al.  1987).  The  probability  that  the  event  occurs  before  t  is  this  number  x  the  probability  that  the 
algorithm  creates  an  event  at  all,  which  is  1  —  e_AA*.  The  resulting  cumulative  distribution  for  vhan  is 

P(vhan+  date(G)  <<)=!-  e-x(,-daU(G)) 


We  are  interested  in  the  probability  that  t  is  the  point  the  program  picks  for  the  first  event  to  occur. 
This  value  is  of  course  zero,  unless  we  express  it  as  a  probability  density: 

J  (t-date(G)+h)  _  M  _  e~A (t-datt(G)))  g-A (t~date(G))  _  A(«- dol^O+A) 

lim -  - 


fc—O 


lim 

h— 0  ft 

j  e-A  (l-date(G)+h) 

t™oh(1  ~  e-Ht-datHG))  > 


lim 

h—0 


1  -  e 


—  AS 


=  A 


The  probability  that  our  rule  fires  immediately  after  t  is  then  A  dtfy  =  A \dt.  This  value  does  not  depend 
on  the  particular  timeline  or  event  G,  so  it  is  in  fact  the  P(E  [t  \  3 G,  F . . .),  so 


P{  some  occurrence  of  e  between  (  and  t  +  dt  |  i4(<)  =  dt/d 

QED 

Timelines  are  created  with  START-TIMELIIE: 

START-TIHELIIEO 

Lat  tl=Greate  a  new  tivaline  with  one  aventinstant  with  happenings  (START) 
Forvard-chain( ’ (START) ,tl) ; 
tl 


We  need  an  entry  point  to  the  system  that  allows  a  program  to  ask  about  an  arbitrary  point  in  the  past, 
not  just  the  last  time  point: 
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PAST- RETRIEVE  ( query , t iaepo int , t iael in* ) 

Let  <anses ,newoccs>*TL-RETRIEVE(query , timepoint .timeline) 

Bring-forward-until-clipped(newoccs .last -time-point (timeline ) , 

timepoint .timeline) ; 

anses 


PAST-RETRIEVE  simply  calls  TL-RETRIEVE,  then  makes  sure  any  new  occasions  created  get  clipped  in  the 
right  place,  using  this  subroutine: 


Br ing-f orward-unt il-clipped (pars ister s , endpo int , beg inpoint , t incline ) 

If  null(persisters)  or  endpoint=beginpoint 
then  persisters 
else  let  persisters  = 

Br ing-f ore ard-unt il-clipped (persisters .past (endpoint) , 

beginpoint , tineline) 

Bring-f orward(persisters , endpoint .timeline) 

This  routine  works  by  bringing  persisters  forward  to  past  (endpoint)  recursively,  then  using  Bring-f orward 
to  get  them  from  past  (endpoint)  to  endpoint. 


Theorem  5  If  a  timeline  is  created  by  START-TIMELIIE,  enlarged  with  TIMELIIE-ADVAICE,  and  queried 
by  TIMELIIE-RETRIEVE  and  PAST-RETRIEVE,  then  the  resulting  timeline  state  will  be  properly  estab¬ 
lished,  properly  clipped,  and  drawn  from  the  distribution  specified  by  Definition  5. 


Proof:  START-TIMELIIE  creates  and  initializes  a  timeline  (using  Forward-chain).  Because  Forward-chain 
uses  Conclude-fron-rule,  it  generates  initial  time  points  with  the  correct  statistics  and  tidiness,  by 
an  argument  similar  to  that  in  the  proof  of  Lemma  2.  TIMELIIE-ADVAICE  AND  TIMELIIE-RETRIEVE 
maintain  these  properties  (Lemmas  2,  3  and  4).  PAST-RETRIEVE  introduces  nothing  novel,  so  it  can  be 
shown  to  maintain  them,  too.  QED 


3  The  Actual  Implementation 


The  actual  implementation  enhances  the  idealized  algorithm  in  two  ways:  greater  expressiveness  and 
greater  efficiency. 
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3.1  Enhancements  to  Expressiveness 


The  implemented  system  is  more  expressive  in  several  ways: 


•  _  rules  are  fully  implemented.  The  only  change  is  that  Establish-af  tar  must  call  TL- RETRIEVE 
for  the  antecedent  of  such  a  rule,  at  the  same  time  point  rather  than  its  predecessor. 

•  In  a  rule  consequent,  a  conjunct  can  be  of  the  form  M,  meaning  that  A  becomes  true  and  persists 
for  time  t. 

•  In  rule  antecedents,  it’s  possible  to  escape  to  Lisp  for  special  extensions  and  low-level  computations. 

•  It  is  also  possible  to  escape  to  a  simple  Prolog-style  backward-chaining  system  for  timeless  goals. 
Each  rule  is  of  the  form  B  *—  Ai  A  A2  A  . . .,  where  each  A,  is  handled  by  another  Prolog-style  rule 
or  a  Lisp  handler. 

•  There  is  an  interface  to  Lisp  for  setting  up  the  initial  time  point. 


The  implementation  is  Lisp-based,  so  rules  actually  have  a  Lisp-style  syntax.  A  A  B  . . .  is  actually 
written  (AID  A  B  ...),  as  usual.  Here  is  a  translation  table  for  the  special  symbols  of  the  system, 
where  typically  is  replaced  by  P  and  “|”  is  replaced  by  E: 


A.  —  -B 

A/E  B 

B  _H  E\A 
d 

A  .—I  E 
A  —  B 

-* A  (in  antecedent) 
->A  (in  consequent) 
tA 

A  A  B  . . . 
p(a,6, ...) 

Variable  v 


(P<-P  A  B ) 
(E->P  A  E  r  B) 
(P<-E  r  B  E  A) 

(P->E  A  d  E) 
«-  A  B) 

(THIOT  .4) 

(CLIP  A) 
(PERSIST  t  A) 
(AID  .4  B  ...) 

(P  a  b  . . . ) 

?v 


Rules  are  asserted  by  the  use  of  the  construct  (DEF-FACT-GROUP  name  — rules — ).  There  is  currently 
only  one,  global,  database  of  temporal  rules  allowed. 

The  presence  of  PERSIST  complicates  TIMELIIE-ADVAICE  somewhat.  Each  occasion  must  have  an  op¬ 
tional  “expiration  date”  when  it  will  cease  to  be  true.  Before  Pass-tia*  can  be  called,  the  system 
must  check  for  occasion  expirations  during  the  time-passage  interval.  If  any  are  found,  it  must  insert  an 
expiration  event,  then  pass  the  time  before  the  expiration,  then  pass  the  time  after  the  expiration. 

It  is  quite  valuable  to  be  able  to  escape  to  Lisp.  Besides  being  more  efficient  for  doing  arithmetic  and 
such,  it  allows  us  to  transcend  the  basic  formalism  in  several  ways.  Here  is  a  list  of  some  of  the  more 
useful  extensions  implemented  this  way: 
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•  (THIOT  p):10  TL-RETRIEVE  doesn’t  actually  have  to  check  for  this  case  specially.  It  just  does  a 
general  check  for  Lisp  handlers,  and  THIOT  has  one  that  tries  to  prove  p,  hoping  to  fail. 

•  (OR  A  B  ...):  Disjunction 

•  (EVAL  exp  tal):  Evaluate  exp  and  unify  result  with  val. 

•  (<  m  n),  (>  m  n),etc.:  Inequalities 

•  (LISP-PRED  r  — args — ):  Run  Lisp  predicate  r  (e  g.,  MEMBER)  on  the  instantiated  arga.  Succeeds 
if  predicate  returns  #T. 

•  (START-TIME  p  <):  Succeeds  ifp  is  true  now  and  started  at  dated  t. 

•  (RECEITLY  p  b  e):  Succeeds  if  p  is  not  true  now,  but  on  the  last  occasion  of  p,  it  started  at  date 
b  and  ended  at  date  e. 

The  presence  of  _  and  EVAL  allows  us  to  express  a  sort  of  rule  that  is  missing  in  the  formal  theory 
developed  in  Section  2  and  Appendix  A.  Suppose  we  want  to  express  the  idea  that  if  a  box  contains  a 
certain  collection  of  objects,  and  one  is  grabbed,  then  a  random  object  leaves  the  box.  We  can  do  that 
thus: 


(P<-E  1.0  (AID  (COITEITS  ?B0X  ?IEV) 

(CLIP  (COITEITS  ?B0X  ?L))) 

(GRAB-FROM  ?B0X) 

(AID  (COITEITS  ?B0X  ?L) 

(EVAL  (RAIDELT  ?L)  ?0B) 

(EVAL  (DELETE  ?0B  ?L)  ?IEV))) 

This  rule  states  that  when  GRAB-FROM  occurs,  the  contents  of  the  box  change  from  ?L  to  ?IEW,  where 
?IEW  is  ?L  with  an  object  removed.  The  choice  of  removed  object  is  made  by  the  Lisp  procedure  RAIDELT. 
In  this  way  we  can  eliminate  a  deficiency  in  the  formalism  of  this  paper,  which  has  trouble  expressing 
the  idea  that  an  event  can  have  one  of  several  mutually  exclusive  outcomes.  Other  formalisms  (e.g., 
that  of  Thiebaux  and  Hertzberg  1992)  allow  for  this  possibility,  but  only  in  the  case  where  the  possible 
outcomes  can  be  laid  out  as  alternative  rules;  I  don’t  know  of  any  formalism  that  allows  for  the  kind  of 
choice  among  alternatives  computed  when  the  rule  is  applied  that  the  rule  above  exemplifies.  Here  we 
achieve  that  choice  only  by  escaping  to  Lisp,  and  thus  leaving  the  formal  semantics  behind. 

There  are  other  applications  of  this  ability  to  generate  random  numbers  in  Lisp.  Suppose  we  want  a 
non-Poisson  model  of  the  occurrences  of  event  type  E.  Suppose  L  is  a  Lisp  function  that  generates  a 
random  interevent  interval  according  to  this  model.  We  introduce  a  proposition  S  (for  “suppress  ET) 
such  that  E  occurs  immediately  whenever  S  is  false: 

10This  negation-as-failure  operator  was  originally  named  by  Sussman  et  al.  in  1971. 
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(DEF-FACT-GROUP  IOI-POISSOI 


(P->E  (TOOT  5)  0  E ) 

(E->P  (EVAL  (L)  ’LIFETIME)  (START)  1  (PERSIST  ’LIFETIME  S)) 

(E->P  (EVAL  (L)  ’LIFETIME)  E  1  (PERSIST  ’LIFETIME  5))) 

The  last  expressiveness  enhancement  I  discuss  in  this  section  is  one  of  the  most  important.  A  key 
desideratum  for  a  temporal-inference  system  is  that  it  connect  smoothly  to  users’  representations  for 
what’s  true  in  the  present.  If  a  program  is  projecting  the  future,  then  its  beliefs  about  the  present  have 
to  be  the  starting  point.  Picture  a  timeline  whose  initial  state  corresponds  to  that  set  of  beliefs.  One 
way  to  make  this  picture  a  reality  is  to  require  all  users  to  represent  the  present  as  a  “short  timeline” 
consisting  of  just  one  instant.  But  there  are  two  reasons  to  reject  this  idea.  First,  it  could  be  clumsy 
and  constraining.  Users  design  their  representations  with  various  goals  of  efficiency  and  expressiveness 
in  mind  that  have  nothing  to  do  with  temporal  inference,  and  we  want  them  to  feel  free  to  focus  on  those 
goals.  Second,  some  of  the  information  about  the  present  may  be  uncertain.  The  program  might  know 
that  there  are  between  2  and  4  balls  in  boxl.  Our  strategy  for  probabilistic  projection  is  to  generate 
definite  projections  with  different  probabilities.  So  if  the  temporal-inference  system  generates  several 
projections,  we  would  like  it  to  pick  2  balls  in  about  1/3  of  the  projections,  3  balls  in  about  1/3,  and  4 
balls  in  about  1/3.  In  each  case,  there  is  no  uncertainty  in  the  initial  state  of  the  timeline. 

What  we  need  is  an  interface  between  the  timeline  manager  and  static  representation  systems.  We 
provide  it  with  a  mutant  rule  of  the  form 


(IIITIALIZER  A  (/  — args — )) 

where  A  is  an  atomic  formula  and  /  is  a  Lisp  function.  To  compute  instances  of  A  true  in  the  initial 
state,  /  is  called  on  the  given  args,  which  are  usually  just  variables  bound  in  A.  f  is  expected  to  return 
a  list  of  the  form  ((pi  li )  (p2  fj)  . . . ),  where  each  p,  is  an  atomic  formula  and  each  U  is  a  lifetime, 
i.e.,  a  number  or  #F.  An  occasion  with  the  given  atomic  formula  will  be  created  starting  in  the  initial 
state  with  the  given  lifetime  (#F  means  “until  clipped”).  The  p,  do  not  have  to  be  instances  of  A. 

The  key  feature  is  that  a  given  /  will  be  called  just  once  for  a  given  set  of  args.  /  may  use  random 
numbers  to  set  the  state  up. 

Here  is  how  we  could  handle  the  situation  in  which  there  are  an  unknown  number  of  balls  in  boxl. 
Suppose  that  we  represent  the  number  of  balls  in  a  box  by  storing  a  pair  “(/  h )”  on  the  property  list 
each  box’s  name,  where  the  actual  number  of  balls  lies  between  /  and  h.  Then  we  could  write  the 
following  initialization  code: 


(DEF-FACT-GROUP  BALL-BOX-IIIT 

(IIITIALIZER  (II  ’X  ?B)  (FILL-BOX  ?B))) 
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(DEFUI  FILL-BOX  (B) 

(LET  ((BUNS  (GET  B  ’BUMBALLS))) 

(COID  ((BULL  BUMS)  ’()) 

(T 

(LET  ((L  (CAR  BUMS))  (B  (CADR  BUMS))) 
(DO  ((B  (+  L  (RABDOM  (♦  1  (-  H  L)))) 
(-  B  1)) 

(RES  ’())) 

((=  B  0)  RES) 

(LET  ((BALLBAME  (GEBSYM))) 

(PUSH  ‘ ((IB  .BALLBAME  ,B)  #F) 
RES)  ))))  ))) 


This  example  is  overly  simple  and  contrived,  but  you  get  the  idea.11 

I  close  this  section  by  explaining  how  the  theory  of  Section  2.1  is  really  expressed  in  the  system: 


(DEF-FACT-GROUP  EXAMPLE-THEORY 


;  Initially,  MUGGET1  is  at  10,0,  and. . . 

(E->P  (TRUE)  (START)  1.0  (LOC  BUGGET1  (COORDS  10  0))) 
;T1000,  an  enemy  robot,  is  at  0,0 

(E->P  (TRUE)  (START)  1.0  (LOC  T1000  (COORDS  00))) 

;  Roughly  every  10  sconds,  T1000  moves  right. 

(P->E  (TRUE)  10.0  (MOVE  T1000  10)) 


;  When  an  agent  moves,  its  location  changes. 

(E->P  (AMD  (LOC  ?B  (COORDS  ?X  ?Y)) 

(EVAL  (+  ?X  ?DX)  ?X1) 

(EVAL  (+  ?Y  ?DY)  ?Y1)) 

(MOVE  ?B  ?DX  ?DY) 

1.0 

11  One  (implication  is  that  in  the  actual  system,  an  initializer  must  also  return  a  “justification”  for  its  answer.  The 
justification  system  is  not  mature  enough  to  talk  about  in  this  paper. 


28 


UlD  (LOC  ?B  (COORDS  ?X1  ?Y1)) 

(CLIP  (LOC  ?B  (COORDS  ?X  ?Y))  ))) 


;  Whenever  TIOOO  and  the  nugget  are  in  the  same  place, 
;T1000  grasps  the  nugget,  typically  within  1  second. 
(P->B  (AID  (LOC  TIOOO  (COORDS  ?X  ?Y)) 

(LOC  IUGGET1  (COORDS  ?X  ?Y))) 

1.0 

(GRASP  TIOOO  VUGGETD) 


;  Grasping  succeeds  with  a  probability  of  80%. 

(P<-E  0.8  (AID  (HOLDIIG  ?A  ?B) 

(CLIP  (LOC  ?B  (COORDS  ?X  ?Y)))) 
(GRASP  ?A  ?B) 

(AID  (LOC  ?A  (COORDS  ?X  ?Y)) 

(LOC  ?B  (COORDS  ?X  ?Y))))) 


3.2  Optimizations 


A  key  observation  about  temporal  inference  is  that  many  deductions  are  repeated  several  times.  The 
procedure  Eatablish-qusry  runs  .♦— |  rules  at  a  time  point,  but  also  calls  itself  recursively  to  run  them 
at  all  previous  time  points,  because  a  query  instance  that  becomes  true  earlier  could  persist  until  now. 
Hence  the  unifications  in  Establish-after  will  be  repeated  many  times.  Similarly,  Check-clipping 
will  be  called  every  time  a  new  event  is  added,  and  it  will  call  Been-clipped  on  every  occasion  in 
ovexTap?ing-occasion(last-tima-point(ttme/me)),  and  run  through  the  |  rules.  The  same  sort 
of  pattern  recurs  in  typical  applications  of  the  timeline.  A  planning  algorithm  might  issue  the  request 
(LOC  AvSST  ?W)  repeatedly. 

Another  key  observation  is  that  most  queries  start  at  the  end  of  a  timeline.  The  reason  is  that  the 
timeline  is  built  by  repeated  calls  to  TIMELIIE-ADVAICE,  which  calls  TL-RETRIEVE  on  the  antecedent 
of  every  |— »_  rule.  Furthermore,  most  occasions  true  at  the  end  of  the  timeline  remain  true  when  new 
events  are  tacked  on,  and  they  have  to  be  copied  to  the  new  eventinstant.  Many  of  these  queries  end  up 
being  propagated  into  the  past,  but  action  in  the  present  dominates. 

On  top  of  this,  the  idealized  versions  of  the  algorithms  give  no  details  as  to  how  the  system  is  to  perform 
all  the  retrievals  of  rules  and  occasions  that  are  called  for.  The  idealized  versions  also  do  not  handle 
PERSIST  literals,  rules,  or  IlITIALIZERs. 

To  meet  all  these  challenges,  the  implemented  algorithm  differs  from  the  idealized  version  in  several 
respects: 
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1  When  using  and  j  rules,  TIMELIIE-RETRIEVE  does  not  bother  to  check  that  the  annihilation  of 
the  consequent  is  true  before  the  event.  In  many  cases,  the  check  would  be  redundant,  and  usually  the  rule 
writer  will  not  mind  making  the  check  explicit  when  necessary.  Because  of  this  optimization,  it  is  possible 
for  there  to  be  two  overlapping  occasions  with  the  same  proposition,  which  is  formally  meaningless.  That 
means  that  a  program  that  queries  the  timeline  may  have  to  be  careful  about  duplicate  answers.  However, 
one  thing  we  do  not  have  to  worry  about  is  the  possibility  that  the  multiple  occasions  will  trigger  later 
|— ♦_  rules  multiply,  causing  two  overlapping  occasions  to  yield  four  consequences,  which  then  yield  eight, 
and  so  forth.  That’s  because  the  first  time  an  occasion  is  created  (by  Conclnds-f  roa-ruls)  starting  at 
a  eventinstant,  an  entry  is  made  for  its  proposition  in  the  established-queries  table  for  the  eventinstant, 
so  Concluda-from-rule  will  avoid  creating  a  second  one  starting  at  the  same  eventinstant. 

2  Occasions  are  not  stored  in  lists,  but  discrimination  trees  (Charniak  et  al.  1987).  I  call  these  formula 
trees.  Given  a  formula  with  free  variables,  one  can  efficiently  retrieve  from  a  formula  tree  all  the  occasions 
it  contains  whose  propositions  match  that  formula. 

3  Rules  are  also  indexed,  albeit  somewhat  less  efficiently.  Different  kinds  of  rules  are  indexed  as  they 

will  be  retrieved,  each  associated  with  one  or  more  symbols.  |— *.  (E->P)  rules  are  accessible  from  the 
main  functor  of  their  events,  because  they  will  be  used  in  Forward-chain  just  after  an  event  has  been 
created.  |  (P<-E),  _  (P<-P)  and  <—  (<-)  rules  are  indexed  by  each  predicate  in  their  consequents. 

*|  (P~>E)  rules  are  left  unindexed,  because  they  are  all  tried  every  time  the  timeline  is  extended. 

In  each  case,  the  system  actually  stores  a  pointer  from  the  indexing  symbol  to  a  list  of  all  fact-group 
names  that  contain  a  relevant  rule.  The  reason  for  this  is  to  simplify  redefinition  of  fact  groups. 

4  A  timeline,  formerly  just  a  list  of  eventinstants,  acquires  several  new  fields,  and  now  looks  like  this: 

•  FIRST:  A  pointer  to  the  time  point  for  the  START  eventinstant. 

•  LAST:  A  pointer  to  the  iast  time  point  (so  far). 

•  IIDEFIIITE-PERSISTERS:  A  formula  tree  containing  all  occasions  that  overlap  with  the  last  event¬ 
instant  of  the  timeline. 

•  EARLIEST-EXPIRER:  The  date  when  the  next  occasion  in  IIDEFIIITE-PERSISTERS  will  expire,  or 
#F  if  all  of  them  will  persist  until  clipped. 

•  EXPIRERS:  A  list  of  ail  occasions  in  IIDEFIIITE-PERSISTERS  with  definite  expiration  dates. 

•  QUERIES:  A  table,  indexed  by  predicate,  of  all  queries  that  have  ever  been  issued  for  this  timeline. 

•  EVEIT-RULES:  All  the  .— *|  rules,  collected  once  from  fact  groups  when  the  timeline  is  initialized. 

The  QUERIES  table  can  be  used  to  “uniquify”  queries.  That  is,  when  a  query  comes  into  the  system, 
the  first  action  is  to  see  if  a  variant  of  this  query  has  been  handled  before.  If  not,  this  one  is  stored  in 
the  table.  If  a  variant  is  found,  TL-RETRIEVE  uses  the  stored  variant  instead  of  the  original  query.  This 
tactic  allows  us  to  use  EQ  to  test  for  whether  a  query  is  established,  rather  than  a  much  more  expensive 
variant  test  (which  needs  to  happen  just  once). 

The  system  uses  the  same  table  to  uniquify  occasion  propositions,  for  much  the  same  reason. 

5  An  eventinstant  actually  looks  like  this: 
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>  OATS:  When  it  happens 

>  TIMESTAMP:  A  finer-grained  time  measure,  used  by  applications.  (Two  consecutive  eventinstants 
can  have  the  same  DATE,  but  not  the  same  TIMESTAMP.) 

>  HAPPE1IIGS:  Events  that  happen  here;  ()  if  nothing  happens. 

>  BEGIIIERS:  A  formula  tree  containing  occasions  that  start  here  and  are  eventually  clipped. 

>  PERSISTERS:  A  formula  tree  containing  occasions  that  started  before  this  eventinstant  and  are 
clipped  at  some  later  eventinstant. 

>  TEMPS:  A  formula  tree  containg  occasions  derived  from  _  rules.  These  occasions  do  not  persist 
past  the  next  eventinstant. 

>  ESTABLISHED:  A  list  of  queries  that  have  been  established  here.  The  queries  are  uniquified  —  that 
is,  they  all  appear  in  the  QUERIES  table  for  this  timeline  — ,  so  the  system  can  use  EQ  to  search  it. 

>  CLIPIOTES:  As  before,  a  table  of  (atomic-formula,  boolean)  pairs  recording  whether  any  occasion 
with  that  formula  will  be  clipped  at  this  eventinstant.  These  formulas  are  also  uniquified,  so  an 
Eq  test  can  be  used  to  check  the  clip  status  of  an  occasion. 


We  still  have  some  occasions  associated  directly  with  an  eventinstant,  but  only  those  whose  end¬ 
points  have  been  seen.  Those  still  true  at  the  end  of  the  timeline  (so  far)  will  be  stored  in  the 
IIDEFIIITE-PERSISTERS  table  for  the  timeline.  When  a  new  eventinstant  is  added,  clipped  occasions 
must  be  moved  from  this  table  to  the  eventinstants  they  overlap.  But  most  of  the  table  stays  the  same. 

The  procedure  Match-against-occasions  now  must  check  four  sets  of  occasions:  the  BEGIIIERS, 
PERSISTERS,  and  TEMPS  of  the  eventinstant  in  question,  plus  all  elements  of  IIDEFIIITE-PERSISTERS 
that  overlap  the  eventinstant. 

6  As  rules  are  defined,  the  system  keeps  a  table  that  associates  an  event  functor  with  all  the  predicates 
whose  occasions  it  can  possibly  clip  by  way  of  P<-E  rules.  E.g.,  If  (MOVE  . . . )  can  clip  only  occasions  of 
the  form  (LOC  . . . ),  then  the  clip-table  entry  for  MOVE  is  (LOC).  This  table  enables  TIMELIIE-ADVAICE 
to  update  the  IIDEFIIITE-PERSISTERS  formula  tree  efficiently.  Formula  trees  are  discriminated  first  by 
predicate,  so  the  only  subtrees  that  need  to  be  touched  when  an  eventinstant  ( A  . . . )  is  added  to  the 
timeline  are  those  with  a  predicate  in  either  the  clip  table  for  A  or  the  clipnotes  for  the  new  eventinstant. 

7  An  occasion  actually  looks  like  this: 

*  PRQPO:  The  proposition,  uniquified  using  the  QUERIES  table  of  the  timeline. 

*  BEGII:  The  time  point  where  the  occasion  begins. 

*  EID:  If  the  end  of  the  occasion  is  known,  this  is  it.  Otherwise,  this  will  be  a  list  of  “chainrecs,” 
each  of  which  represents  a  |  that  could  clip  it.  See  below. 

*  EXPIRATIOI:  If  the  occasion  has  a  finite  lifetime,  this  is  its  expiration  date.  Otherwise,  this  field 
is  #F. 
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*  JUSTIF:  The  “justification”  for  the  occasion,  a  record  of  how  it  came  to  occur.  In  this  paper,  I  do 
not  discuss  justification  structures,  but  the  idea  is  to  set  up  data  dependencies  to  link  events  with 
their  effects. 


8  To  avoid  repeating  all  those  unifications,  the  system  uses  the  new  datatype  cAainrec  to  keep  track  of  a 
“deduction  in  progress.”  For  example,  whenever  an  occasion  for  proposition  A  is  generated,  the  system 
finds  all  P<-E  rules  that  contain  a  literal  unifying  with  (CLIP  A).  For  each,  it  makes  a  chainrec  (“chain 
record”)  recording  the  rule,  the  substitution,  and  the  result  of  applying  the  substitution  to  the  event  in 
the  rule.  It  then  sets  the  E1D  field  of  the  occasion  to  be  a  list  of  ail  such  chainrecs.  Bssn-clippsd  then 
need  not  consult  the  database,  but  simply  gets  this  list  from  the  occasion  under  consideration.  If  the 
list  is  empty,  then  the  occasion  will  never  be  clipped  by  a  P<-E  rule. 

9  The  other  use  of  chainrecs  is  in  backward  chaining.  The  QUERIES  table  of  a  timeline  is  used  to 
associate,  with  each  query  Q,  a  list  of  chainrecs  for  all  the  P<-E,  P<-P,  and  IIITIALIZER  rules  whose 
consequents  contain  a  literal  that  unifies  with  Q.  Each  such  chainrec  records  the  substitution  resulting 
from  a  successful  unification  of  Q  with  some  literal  in  the  consequent  of  a  rule.  Establish-qusry  must 
then  try  these  rules  at  every  eventinstant. 

Actually,  Establish-qusry  can  do  a  lot  better  than  that,  because  what  the  system  stores  with  each 
uniquified  query  is  actually  two  lists,  the  “start  chainrecs”  and  the  “regular  chainrecs.”  The  former  list 
is  for  IIITIALIZER  rules  and  rules  of  the  form  (P<-E  . . .  (START)  . . . ).  There  is  no  point  in  trying  these 
rules  out  anywhere  except  at  the  beginning  of  the  timeline,  and  hence  no  point  in  running  through  a  deep 
recursion  to  get  there.  These  rules  are  handled  specially  by  jumping  to  (START)  and  then  working  back 
to  the  query  point  using  Bring-f orward  Regular  P<-E  rules  require  repeated  calls  to  Establish-aftsr. 
If  there  aren’t  any,  the  system  skips  this  phase. 


4  Experimental  Results 


In  a  general  rule  interpreter  like  this  one,  it  is  impossible  to  make  general  statements  about  run  times. 
It’s  not  hard  to  make  up  rule  sets  that  require  superexponential  times  to  do  projections.  However,  most 
rule  sets  are  better  behaved.  They  tend  to  give  rise  to  the  same  or  similar  queries  repeated  at  successive 
time  steps,  so  many  attempts  to  answer  them  need  only  scan  back  through  one  eventinstant.  Most  long 
scans  result  from  queries  that  go  back  to  the  START  event  to  transfer  information  from  the  current  world 
model  to  the  timeline. 

Here  is  an  example  of  a  theory.  It  is  contrived,  but  illustrative,  and  not  as  violent  as  some: 
(DEF-FACT-GROUP  IIITIALIZERS 

(P<-E  1.0  (KISSED  0)  (START)  (ALWAYS)) 

(P<-E  1.0  (HUGGED  0)  (START)  (ALWAYS)) 
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(P<-E  i.O  (LOC  ?X  ?X)  (START)  (ALWAYS)) 


(INITIALIZER  (PROTECTED  ?X)  (IIITI ALLY-PROTECTED  ?X))) 

(DEFU1  I1ITIALLY-PROTECTED  (X) 

(COED  ((AID  (IOT  (HASMVARS  X)) 

(RAVDON-CHOICE  0.5)) 

(VALUES  (LIST  ‘(#F  (PROTECTED  ,X))) 

#* (LAMBDA  ()  ’#T))) 

(T  ’())  )) 

(DEF-FACT-GROUP  RAIDOM-EVEVTS 

(P->E  (AID  (KISSED  ?I) 

(EVAL  (+  ?I  1)  ?I1)) 

10.0 

(KISS  ?I1)) 

(P->E  (AID  (HUGGED  ?I) 

(EVAL  (+  ?I  2)  ?I2) ) 

20.0 

(HUG  ?I2))) 

(DEF-FACT-GROUP  KISS-AID-HUG 

(E->P  (KISSED  ?I) 

(KISS  ?J) 

1.0 

(AID  (KISSED  ?J) 

(CLIP  (KISSED  ?I)))) 


(P<-E  0.8 

(PERSIST  50  (BLUSHIIG  ?I  ?J)) 
(KISS  ?J) 

(AID  (HUGGED  ?I) 

(KISSED  ?I) 

(THIOT  (PROTECTED  ?I) ) ) ) 
(E->P  (HUGGED  ?I) 
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(HUG  ? J) 

1.0 

(AID  (HUGGED  ?J) 

(CLIP  (HUGGED  ?I))))) 

(DEF-FACT-GROUP  KICK-AR0U1D 

(E->P  (AID  (IEIGHBOR  ?L  ?*) 

(LOC  ?X  ?I) 

(EVAL  (+  ?■  8)  ?IEW) ) 
(KICK  ?L) 

1.0 

(AID  (LOC  ?X  ?*EW) 

(CLIP  (LOC  ?X  ?!))))  ) 

(DEF-FACT-GROUP  IEIGHBOR-DEF 

(<-  (IEIGHBOR  ?L1  ?L2) 

(EVAL  (-  ?L1  2)  ?L2)) 

(<-  (IEIGHBOR  ?L1  ?L2) 

(EVAL  (-  ?L1  1)  ?L2) ) 

(<-  (IEIGHBOR  ?L1  ?L2) 

(EVAL  (+  ?L1  1)  ?L2)) 

(<-  (IEIGHBOR  ?L1  ?L2) 

(EVAL  (+  ?L1  2)  ?L2) )  ) 


There  are  two  types  of  autonomous  events,  (KISS  i)  and  (HUG  i).  Initially  we  have  (KISSED  0)  and 
(HUGGED  0);  every  5  seconds  (on  the  average)  a  KICK  event  occurs,  which  increments  (KISSED  t)  to 
(KISSED  i  +  1).  Similarly  for  HUGGing,  which  occurs  every  10  seconds  on  the  average,  but  increments 
(HUGGED  i)  by  two.  If  (HUGGED  t)  and  (KISSED  i)  are  ever  both  true  when  a  (KISS  j)  occurs  and 
(PROTECTED  i)  is  not  true,  then  an  occasion  of  (BLUSHIIG  i  j)  begins  and  persists  for  50  seconds. 
Objects  are  initially  PROTECTED  with  probability  1/2. 

We  test  the  performance  of  the  program  by  generating  exogenous-event  streams  consisting  of  N  +  1 
events  of  the  form  (KICK  5),  (KICK  15),  . . .,  (KICK  5  +  ION).  As  N  increases,  the  initial  state  grows 
to  generate  more  and  more  occasions  of  the  form  (LOC  x  x).  Here  is  a  table  of  the  execution  times  that 
result: 
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Seconds 

MTtfVCJW 


OJO 


OJO 


•  • 


•  • 


100  300 

Number  of  events 


300 


Figure  3:  Time  Per  Event  Required  to  Generate  N  Events 


N 

Run  time 

Number  of  events 

Time/event 

10 

1.0 

28 

0.03 

20 

3.4 

54 

0.06 

30 

6.2 

81 

0.08 

40 

8.8 

101 

0.09 

50 

11.8 

128 

0.09 

60 

15.0 

154 

0.10 

70 

19.0 

183 

0.10 

80 

22.9 

202 

0.11 

90 

27.5 

230 

0.12 

100 

31.7 

249 

0.13 

These  numbers  are  obtained  by  running  each  experiment  10  times  and  averaging  the  results.  The  key 
column  is  Time/event.  This  grows  linearly  with  N,  indicating  that  the  total  time  to  generate  a  timeline 
with  N  exogenous  events  is  quadratic  in  N.  (See  Figure  3.)  It’s  hard  to  imagine  doing  much  better 
than  that,  given  that  generating  a  timeline  takes  many  retrievals  in  order  to  run  |— *  _  rules,  and  these 
retrievals  involve  a  linear  component,  namely,  the  time  required  to  check  where  an  occasion  added  to 
the  beginning  of  the  timeline  gets  clipped. 


5  Conclusions 


I  have  presented  a  formalism  for  expressing  probabilities  of  outcomes  of  events,  and  shown  that  it  is 
possible  to  use  rules  expressed  in  this  formalism  to  generate  scenarios  —  “projections”  —  of  what  will 
result  from  a  given  event  sequence.  The  formalism  has  a  formal  semantics,  and  I  have  proven  two  main 
results  about  it: 


1.  A  broad  class  of  rule  sets  expressed  in  this  formalism  yield  well  defined  models. 

2.  The  projection  algorithm  generates  a  particularlprojection  with  the  probability  specified  by  the 
formal  semantics. 
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Furthermore,  the  algorithm  appears  to  be  “efficient.”  The  running  times  given  in  Section  4  are  not  in 
themselves  impressive,  but  I  hope  the  growth  rate  is.  As  the  number  of  events  projected  grows,  the  time 
to  generate  one  more  event  grows  linearly,  and  slowly.  Presumably  translating  the  algorithm  from  Lisp 
to  C  would  produce  a  big  absolute  speedup,  although  we  have  no  plans  to  do  that. 

This  temporal-projection  system  is  now  in  use  in  the  XFRM  planning  system  being  developed  at  Yale. 
It  is  available  to  interested  parties  by  ftp;  please  contact  mcdermott@cs.yale.edu. 

Some  enhancements  to  the  system  would  obviously  be  worth  exploring.  The  present  formalism  allows 
random  events  to  happen  at  arbitrary  real  times,  but  it  does  not  allow  for  continuous  change.  For 
example,  we  might  want  to  specify  that  while  an  agent  is  moving,  its  fuel  supply  depletes  continuously 
with  a  random  slope.  It’s  not  hard  to  see  how  to  alter  the  implementation  to  allow  for  such  phenomena, 
and  in  fact  we  have  done  this  on  an  ad-hoc  basis.  If  a  quantity  is  changing  continuously  toward  some 
threshold  (e.g.,  an  empty  fuel  tank),  then  have  Pass-time  check  to  see  if  it’s  gotten  there  during  the 
time-passage  interval  being  filled  in.  If  it  has,  then  an  eventinstant  declaring  the  occurrence  of  a  “reached 
threshold”  event  can  be  added  to  the  timeline.  (This  is  very  similar  to  what  happens  with  expirations 
of  facts  with  finite  lifetimes.) 

Unfortunately,  as  soon  as  we  allow  continuous  change,  the  formal  analysis  will  have  to  be  changed 
significantly.  The  proofs  in  the  appendix  depend  crucially  on  nothing  happening  between  eventinstants, 
and  that  assumption  will  have  to  be  relaxed.  Possibly  the  techniques  of  Penberthy  (1993)  will  help. 

Another  class  of  extensions  revolves  around  allowing  rules  to  make  random  choices  among  sets  of  pos¬ 
sibilities.  Currently,  we  can  arrange  for  that  only  by  escaping  to  Lisp  to  generate  random  numbers, 
and  it  would  be  better  if  the  choice  was  indicated  in  the  rule,  and  governed  by  a  formal  analysis.  The 
problem  here  is  not  the  formal  analysis  itself,  but  figuring  out  a  framework  that  encompasses  all  the 
possibilities.  One  approach  might  be  to  introduce  a  new  sort  of  rule  that  allows  for  labeled  random 
choices  at  a  point.  (They  have  to  be  labeled  so  that  different  rules  can  refer  to  different  random  choices 
at  the  same  eventinstant.)  The  choice  of  whether  a  rule  fires  or  not  is  then  just  a  special  case. 
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A  A  Rigorous  Definition  of  Timeline  Probability 


In  this  appendix,  I  will  go  back  and  make  sure  that  Definition  5  makes  sense,  by  showing  that  it  can 
be  phrased  in  terms  of  the  limit  of  a  series  of  finite  probability  distributions.  Throughout  this  section, 
I  will  assume  that  a  set  of  temporal  rules  is  simple  in  the  sense  explained  by  this  definition: 


Definition  8  A  set  of  rules  is  simple  if  and  only  if 
1.  There  are  no  _  rules. 
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2.  There  is  a  positive  number  6  such  that  if  a  probability  r  occurs  in  a  rule  instance,  r  (0, 6)  and 
r  g  (1  —  delta,  1).  That  is,  if  r  is  not  0  or  1,  it  is  in  the  interval  [6, 1  -  $]. 

3.  There  is  a  positive  number  D  such  that  if  a  duration  d  occurs  in  a  rule,  d  <  D. 


All  of  these  clauses  are  for  technical  convenience.  The  first  clause  spares  us  from  having  to  invest  much 
worry  about  the  static  component  of  the  theory.  In  particular,  we  can  avoid  talking  about  stratification 
and  other  tactics  for  taming  recursive  backward-chaining  rules  (Lifschitz  1987).  There  is,  as  far  as  I 
know,  no  special  technical  problem  associated  with  -  * —  -  rules  in  a  temporal  context.  The  next  two 
clauses  do  not  place  a  severe  limitation  on  the  expressive  power  of  the  language,  and  allow  us  to  draw 
various  useful  conclusions.  For  example,  if  an  infinite  set  of  rule  instances  have  probability  /  1,  the 
product  of  their  probabilities  is  0  (=  lim,t_oo(l  -  6)k) 

The  discrete  analog  of  a  world  is  a  finite  state  series 


Definition  9  A  finite  state  series  is  a  mapping  from  integers  to  pairs  (s,  E) ,  where  s  is  a  world  state 
and  E  is  an  event.  The  domain  of  a  finite  state  series  is  a  finite  set  of  integers  {0, 1, ...,/—  1),  and  1  is 
said  to  be  the  length  of  the  series. 


If  the  ith  element  of  a  finite  state  series  is  (s,  E),  then  that  means  that  at  the  *th  time  instant  the  world 
is  as  described  by  s,  and  that  the  next  event  to  occur  is  E.  There  is  a  distinguished  event  type  <t>  that 
means  “nothing  happens,”  and  another,  ft,  that  means  “the  world  ends.”  Most  time  points  have  event 
<t>.  <t>  is  not  an  element  of  Q,  the  set  of  event  types,  and  no  rules  refer  to  it. 


Definition  10  An  L,  pi-series  for  a  T,C  pair,  a  real  number  L  such  that  duration(C)  <  L,  and  a  real 
number  p  >  0,  where  p  <  the  smallest  interval  between  two  consecutive  elements  of  C,  is  a  finite  state 
series  5  of  length  /  =  \L/p] ,  such  that  the  first  element  of  the  series  has  s  =  F;  and  if  if  Cy  =  E\  t ,  then 
=  (s,  E)  for  some  state  s. 


Though  we  do  require  that  every  L,  p  series  have  F  as  first  state,  we  do  not  require  that  it  have  START  as 
first  event.  It  turns  out  to  be  technically  simpler  just  to  let  the  first  event  in  C  play  the  role  of  initializer, 
and  by  convention  to  have  it  always  be  START. 

In  addition  to  making  time  discrete,  we  are  going  to  want  to  talk  about  particular  kinds  of  sets  of  worlds, 
revolving  around  this  definition: 


Definition  11  A  partial  state  is  the  set  of  all  world  states  that  satisfy  a  (possibly  infinite)  set  of 
constraints  of  the  following  two  sorts: 


1.  Assignments  of  a  truth  value  (#T,  #F,  or  i.)  to  a  particular  proposition  symbol; 

2.  Assignments  of  the  value  #T  to  a  non-singleton  disjunction  of  literals. 

If  an  assignment  of  the  first  kind  is  in  the  set,  then  the  proposition  in  question  is  said  to  be  constrained 
by  the  partial  state.  A  finite  partial  state  is  one  that  constrains  only  a  finite  set  of  propositions.  An 
assignment  of  the  second  kind  is  called  a  disjunctive  constraint,  and  it  has  a  secondary  role,  which  will 
be  explained  later. 
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For  example,  suppose  that  V  =  {A,B,C,D}.  There  is  a  partial  state  with  four  states,  all  of  which  specify 
that  A  is  true  and  B  is  false,  such  that  each  combination  of  assignments  to  C  and  D  is  represented.  This 
partial  state  has  no  disjunctive  constraints,  and  so  we  can  represent  it  as  a  function  from  propositions 
to  the  set  {#T,  #F,  ?,  1}: 

s(A)  =  #T 
s(B)  =  #F 
-(C)  =  ? 

-(D)  =  ? 

Hence,  even  though  partial  states  are  technically  sets  of  world  states,  I  will  use  the  letter  s  to  denote 
partial  states,  and  will  evaluate  propositions  with  respect  to  them.  In  such  a  context,  I  will  take 
s(x  A  ?)  =  ?,  unless  x  —  L.  I  will  let  S  stand  for  the  set  of  all  possible  world  states,  which  is  the  same 
as  the  partial  state  in  which  all  propositions  are  ?. 

Now  suppose  we  add  a  disjunctive  constraint  that  assigns  value  #T  to  C  V  ->D.  We  can  think  of  this  as 
removing  from  our  example  partial  state  the  world  state  that  assigns  #F  to  C  and  #T  to  D.  However,  I 
will  introduce  such  constraints  only  in  contexts  where  the  measure  of  the  eliminated  world  state  can  be 
shown  independently  to  be  zero.  Hence,  I  will  require 

Principle  V  (for  vacuity):  The  measure  assigned  to  a  partial  state  with  disjunctive  constraints  is  ex¬ 
actly  the  same  as  the  measure  assigned  to  the  same  state  w’th  one  or  all  of  the  disjunctive  constraints 
eliminated. 

To  put  it  another  way:  even  though  a  partial  state  with  nonzero  probability  may  include  world  states  that 
make  arbitrary  assignments  to  these  propositions,  these  world  states  may  themselves  have  probability 
zero,  so  that  constraining  them  away  with  disjunctive  constraints  does  not  alter  the  probability  of  the 
partial  state. 

Hence  we  can  continue  to  think  of  partial  states  as  functions  from  propositions  to  the  set  {#T,  #F,  X,  ?}, 
and  by  extension  as  functions  from  formulas  to  this  set,  but  the  presence  of  disjunctive  constraints 
may  bring  about  a  situation  where  a  non-?  value  is  assigned  to  a  compound  formula  even  though  only 
?s  are  assigned  to  its  constituents.  In  our  modified  example,  we  will  still  have  s(C)  =  s(D)  =  ?,  but 
s(-iC  A  D)  =  #F. 


Definition  12  Two  partial  states  and  s 2  disagree  if  and  only  if  there  is  a  proposition  symbol  A 
constrained  by  both  such  that  si(A)  ^  S2M).  A  partial  state  S2  extends  a  partial  state  if  and  only  if 
s2(A)  =  «i(A)  whenever  si(A)  ^  ?. 


Next  we  define  a  measure  function  on  L,ji  series.  (Actually,  I  will  not  show  it  is  a  measure  until  after 
some  further  lemmas.) 


Definition  13  An  L,ji-model  for  T  ,C  is  a  pair  (W„,  M^),  where  Wp  is  the  set  of  all  L,ji-senes  for  T,C 
and  Mp  is  a  “measure”  on  WM  that  obeys  a  set  of  constraints  that  are  discrete  analogues  of  those  of 
Definition  5.  In  these  constraints,  we  write  E  [  i  to  mean  the  boolean  random  variable  true  just  for  any 
L,  ^-series  S  in  which  5,  =  (s,  E)  for  some  s;  if  A  is  a  proposition  symbol  we  write  A@i  to  mean  the 
boolean  random  variable  true  just  for  any  L,ji- series  S  in  which  Si  =  (s,  E),  where  E  is  some  event  and 
s(A)  =  #T;  and,  if  s  is  a  partial  state,  we  write  s@i  to  mean  the  boolean  random  variable  true  just  for 
any  p-series  in  which  Si  =  (s',  E)  for  some  s'  £  s  and  some  event  E.  In  the  following  constraints,  the 
letter  s  is  used  to  refer  to  partial  states;  the  letter  E,  to  events: 
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1.  For  all  propositions  A  in  V ,  Mn(A@ 0)  =  0. 

2.  If  E  =  Q  then  AfM(s@(»+l)  |  s'@i  AE  [  »')  =  1  if  *  =  X  and  0  otherwise.  If  E  ^  ft,  then  to  compute 
AfM(s@(i  +  1)  |  s'@i  A  E  [  i),  first  let  ft  be  a  subset  of  the  set  of  all  firable  rule  instances  {Rj}  of 

rj  n  _ 

the  form  Aj/E  |— ►.  fl,  and  Bj  _*— |  E\Aj ,  where  each  s'(.Aj  A  Bj)  =  # T.  Let  Q  be  the  set  of 
all  such  rule  instances  that  might  be  firable,  that  is,  the  maximal  set  with  s'(Aj  A  Bj)  £  {#T,  ?}, 
with  (JO  R  —  0.  Let  U  be  the  set  of  all  rule  instances  whose  firability  is  undefined,  that  is,  those 
for  which  s'(Aj  A  Bj)  =  X.  Treat  each  Bj  as  a  set  of  literals.  If  s  can  obtained  from  a  subset  ft+ 
of  R  by  adding  and  deleting  consequents  of  rules  in  5,  that  is,  if 


X  iff  b  or  -’6  €  Bj  for  some  Rj  £  U 

or  Bj  and  B*  contain  b  or  ->6  for  distinct  Rj  and  ft*  G  R 
?  if  s(6)  ^  X  and  6  or  ->b  £  Bj  for  some  Rj  £Q 

#F  if  s(b)  £  {X,  ?}  and  ->b  £  Bj  for  exactly  one  Rj  £  ft+ 

# T  if  s(6)  £  {X,?}  and  b  £  Bj  for  exactly  one  Rj  £  ft+ 

s'(b)  otherwise 


then 

+  1)  |  s't&i  A  E)i)  =  JJ Pj 
] 

where  pj  =  rj  if  Rj  £  ft+  and  pj  =  1  —  rj  if  Rj  £  R+ .  If  s  cannot  be  derived  from  such  an  ft+, 
then 


Adw(s@(i  +  l)|5'@iAEli)=  < 


Af/i(s+'§(i  +  1)  |  s'@iAEH) 

where  s+  is  the  minimal  extension  of  s 
that  is  derivable  from  an  ft+ 

0  if  there  is  no  extension  s+  derivable  from  an  ft+ 


3. 


The  computation  of  M^E  i  i  |  s@i)  splits  into  two  basic  cases,  depending  on  whether  E  f  i  is 
exogenous.  If  i  =  [date{Ch)/p\  for  some  k,  then  Ad(J(£’  i  i  |  s@»)  =  1  if  E  =  event (C*),  else  0. 
The  endogenous  case,  when  i  ^  [date(Ck)/p\  for  any  k,  is  more  complex.  Let  ft  be  the  set  of  all 

di 

rule  instances  of  the  form  Aj  »|  . . .  ?n  T  such  that  s(Aj)  =  #T.  Let  U  be  the  set  such  that 
s(j47)  =  X,  and  let  Q  be  the  set  such  that  s(>4;)  =  ?,  much  as  in  the  previous  clause.  If  U  is 
nonempty,  or  ft  is  infinite,  or  A  =  >  1/a*  then  M^Eii  \  s@i)  =  1  if  E  =  Q,  else  0.  If 

ft  is  finite  and  Q  and  U  are  empty,  and  A  <  l/p,  then  for  every  event  type  E  let  Re  be  the  subset 
dj 

of  ft  of  the  form  . . .  *|  E 


A  e  = 


€Re 


di 


then 


M^{E[i\  s@i)  =  A Ep 
Mfj(d>  l  i  |  s@i)  =  1  -  Xp 

i  i  |  s@i)  =  0 


Furthermore,  if  E\  and  E 2  are  distinct  event  types,  it  is  always  the  case  that  M n{E\  I  iAE 2 1  i)  =  0. 
If  U  is  empty,  and  A  <  l//i,  but  Q  is  nonempty,  then  there  is  no  constraint  on  i  i  |  s@i). 
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R 

R+ 

s 

Probability 

0 

0 

Bj  =  B2  =  Ct  =  ? 

1 

{1} 

0 

b2  =  ? 

1  -  ri 

{1} 

Bi  =  Ci  =  #T,B2  =  ? 

n 

{2} 

0 

Bi  =C,  =? 

1  -  r2 

{2} 

00 

II 

0 

II 

W 

II 

4b 

H 

r2 

{1.2} 

0 

no  change 

(l-r,)(l-r2) 

{1} 

Bi  =  Ci  =  #T 

ri(l  -  r2) 

{2} 

B2  =  #T 

(1  -  ri)r2 

{1,2} 

Bi  =  Ci  =  #T,B2  =  #T 

rlr2 

Table  1:  Partial  States  and  Their  Probabilities 

4.  If  the  conditional  probability  of  random  variable  E  [  i  or  s@i  is  given  by  one  of  the  clauses  above, 
then  the  random  variable  is  conditionally  independent  of  all  other  random  variables  for  times  on 
or  before  i.  To  be  precise,  if,  for  random  variables  a  and  /?,  M^a  |  /?)  is  supplied  by  Clauses  1,  2, 
or  3,  then  for  any  random  variable  7  that  mentions  only  times  on  or  before  times  mentioned  in  0, 

AM*  |/? A 7)  =  I/?) 

Furthermore,  if  disjunctive  constraints  are  added  to  partial  state  s@i  in  accordance  with  Princi¬ 
ple  V,  then  the  conditional  probability  assigned  each  such  state  is  the  same  as  that  assigned  to 
s@i. 


An  example  will  make  these  rules  seem  clear,  I  hope.  Suppose  a  theory  has  just  these  two  rules: 

Bj  A  Cj 

B2 

s'@i  A  E  j  »),  where  s'  is  the  partial  state 
Table  1  shows  the  possible  outcomes  and 
their  probabilities  for  each  choice  of  R  and  R+ .  The  outcomes  are  specified  in  terms  of  the  difference 
between  the  new  state  and  the  old.  The  third  column  of  the  table  gives  all  possible  partial  states  that 
can  happen.  These  are  not  mutually  exclusive.  For  example,  line  2  of  the  table  gives  the  probability  of 
Ai  =  A2  =  #T,  Bi  =  Ci  =  #F,  B2  =  ?,  which  is  the  union  of  the  states  described  in  lines  6  and  8,  and  has 
probability  (1  —  ri)(l  —  rj)  +  (1  —  ri)r2  =  1  —  rj.  That  is,  if  we  don’t  consider  whether  a  rule  fires  or 
not,  we  get  a  partial  state  that  ranges  over  both  possibilities. 

Definition  14  A  theory  T  is  consistent  for  event  sequence  C  if  and  only  if  there  exists  a  6  such  that 
for  every  0  <  ft  <  6,  no  L,  //-model  of  T,  C  has  }  i)  >  0  for  any  i;  or  A4^(s@i)  >  0  for  any  s  such 

that  s(A)  =  JL  for  some  A.  T  is  consistent  if  it  is  consistent  for  all  C. 

Note  that  the  following  theory  is  inconsistent: 

1 

START 

2- 

P(i)  _—| 

P(.)/E  |i_ 


P(l) 

E 

P(*'+  I) 


Rule  1:  ki/Z  |— . 
Rule  2:  A2/E  |-+_ 

and  suppose  that  we  are  trying  to  compute  Ad^(s@(i  +  1) 
that  assigns  #T  to  Aj  and  A2  and  #F  to  Bi,  B2,  and  Cj. 
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(where  each  rule  involving  t  stands  for  all  rule  instances  obtained  by  substituting  an  integer  >  0  for  i). 
The  reason  the  theory  is  inconsistent  is  that  the  time  scale  of  the  second  rule  decreases  exponentially, 
so  that  for  any  event  sequence  with  duration  >  1,  no  matter  how  small  (x  gets  there  will  be  worlds  with 
Ag  >  l//i,  so  the  event  fl  will  have  nonzero  probability. 

My  strategy  is  to  define  the  probability  of  a  sequence  of  partial  states  in  continuous  time  by  taking  the 
limit  of  the  result  of  adding  up  the  probabilities  of  all  relevant  event  sequences.  We  have  to  be  careful 
about  the  way  we  count  event  sequences,  in  order  to  avoid  adding  up  an  infinite  number  of  zeros.  I  start 
by  showing  that  the  probability  that  an  infinite  number  of  events  occurs  is  0. 


Theorem  6  If  T  is  consistent,  then  for  arbitrary  occurrence  sequence  C  and  duration  L  >  duration(C), 

lim  MJ\L/fx]  events  occur)  =  0 

f»->0 

where  the  expression  u\L/ix]  events  occur”  is  shorthand  for  “-><£i0  A  -><£  1 1  A  . . .  A  -># [  [L/fi]-l.” 


Proof:  Since  T  is  consistent,  there  is  some  6  such  that  for  fx  <  6,  Ajx  <  1,  where  A  is  as  defined  in 
clause  3  of  Definition  13.  Therefore  AS  <  1  and  A  <  l/S.  Hence  the  probability  that  N  =  \L/fi\  non-<p 
events  occur  is  less  than 


As  fx  — *  0,  this  quantity  — *  0.  QED 


Theorem  6  means  that  we  can  analyze  the  limit  of  M ^  as  /i  —  0  in  terms  of  the  probabilities  of  worlds 
with  a  finite  number  of  occurrences.  Call  each  such  world  a  scenario.  Then  we  would  like  to  define,  for 
any  boolean  random  variable  a, 


M(ot)  =  lim 


£ 

all  scenarios 


Muda]^  |  scenano)M ^(scenario) 


where  [. .  ]li  is  defined  thus: 


Definition  15  The  discretization  [a]^  of  a  formula  involving  random  variables  of  the  form  A\t  and 
Ait  is  the  formula  obtained  by  making  the  following  substitutions  in  a: 

E[t  — *  Ellt/rfr 
Aft  — »  A@[t/n  \fx 
Alt  — ►  A®[t/n\n  +  1 


Unfortunately,  in  the  limit  the  probability  of  most  scenarios  is  zero,  because  the  probability  that  an 
endogenous  event  occurs  at  a  particular  time  is  zero.  To  sum  over  all  scenarios,  we  have  to  group  the 
terms  into  combinations  with  non-infinitesimal  probabilities.  The  appropriate  groups  are  what  I  will  call 
“scenario  schemas.” 


Definition  16  A  T,C,  L  scenario  schema  is  a  finite  sequence  E  of  triples  (£i, . . .  ,£*),  where  each  is 
of  the  form  ( ),  written  Sj  1  Ej  J. tj,  where  sj  is  a  partial  state,  Ej  is  an  event  type,  and  tj  is  a 
unique  free  variable  or  a  date  (a  real  number  >  0  and  <  L).  Those  with  free  variables  are  called  variable 
elements  of  E;  those  with  dates,  fixed.  The  free  variables  will  be  bound,  and  summed  and  integrated 
over,  in  what  follows. 

The  schema  is  intended  to  denote  the  set  of  all  scenarios  in  which  exactly  the  events  Ej  1  tj  occur,  in 
the  given  order,  and  an  element  of  Sj  is  true  just  after  the  occurrence  Ej  i  tj.  We  call  this  the  set  of 
scenarios  described  by  S.  E  must  satisfy  all  of  the  following: 
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No  two  £j  have  the  same  tj; 

and  for  every  E  i  t  in  C,  there  is  a  (j  €  E  such  that  (j  =  Sj  i  E  i  t; 
and  the  fixed  elements  of  E  are  ordered  by  increasing  t} ; 

and  if  there  is  an  element  a  i  E  j  0,  it  is  the  first  element  of  E;  if  there  is  an  element  a  1 E 1 L,  it  is  the 
last  element. 

Definition  17  A  scenario  schema  is  finite  if  and  only  if  each  of  its  partial  states  is  finite. 

We  can  discretize  scenario  schemas: 


Definition  18  The  /i- discretization  of  a  scenario  schema  E  for  a  is  written  [E]p,  and  is  constructed  by 
transforming  every  element  of  E  thus: 

1 .  Replace  a  fixed  ^  of  the  form  s}  j  Ej  l  tj ,  with  Sj  j  Ej  1  [tj /fi\ . 

2.  Replace  a  variable  £j  of  the  form  Sj  [  E3  [  tj  with  Sj  [  Ej  [  ij ,  where  ij  is  a  unique  free  variable  that 
occurs  nowhere  else  in  [E]^. 

[Sj^i  is  said  to  describe  the  set  of  L,  ^-series  in  which  the  Ej  happen  in  the  given  order,  the  sj  ensue, 
and  <t>  happens  at  all  time  points  where  no  Ej  happens. 


The  probability  of  a  discretized  scenario  schema  cannot  be  computed  directly,  but  we  can  define  a  special 
case  that  is  easy  to  compute. 


Definition  19  Let  j  be  a  position  in  a  schema  S  (1  <  j  <  length(E)),  with  triple  8j  1  Ej  [  tj  occurring  at 
that  position.  Let  the  preceding  state  sp  be  F  if  j  =  1,  and  Sj- 1  if  j  >  1.  Let  A(Ri)  be  the  conjunction 
of  the  antecedent  of  rule  instance  Rj  and  the  annihilation  of  its  consequent.  A  T,C,L  scenario  schema 
S  is  closed  at  j  with  respect  to  a  set  of  rule  instances  R  if  and  only  if  there  is  a  subset  R+  of  R  such 
that  for  every  rule  instance  R+  €  R+,  sp(A(Ri))  =  #T;  and  for  every  rule  instance  in  R  —  R+, 
ap(A(R.))  =  #F. 

S  is  closed  at  j  with  respect  to  proposition  symbol  P  if  and  only  if  it  is  closed  with  respect  to  the  set  of 
A/Ej  \—*-B  or  fl_*—  |  Ej\A  rule  instances  with  the  literal  P  or  ->P  occurring  in  B. 

S  is  closed  at  j  with  respect  to  Ej  if  and  only  if  it  is  closed  with  respect  to  the  set  of  all  _— *|  rules  (not 
just  those  for  Ej). 


Note  that  if  E  is  a  scenario  schema  for  a  consistent  theory  closed  at  position  j  with  respect  to  a  proposition 
P,  there  can  be  at  most  one  rule  instance  Arable  in  s;  _j  that  asserts  P  or  -’P;  all  the  others  are  unfirable 
in  any  extension  of  E. 


Definition  20  A  scenario  schema  S  is  closed  at  j,  given  that  the  j’th  element  of  S  is  8j  1  Ej  i  tj,  if  it 
is  closed  at  j  with  respect  to  Ej,  and  with  respect  to  every  proposition  symbol  constrained  by  8j .  A 
scenario  schema  is  closed  if  it  is  closed  at  every  j ,  1  <  j  <  length(E). 
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The  following  theory  will  provide  an  example  of  cloaedness: 


START 

O  \ 

fragila 

START 

0.5 
1— - 

full 

drop 

1 

H- 

onfloor 

Tragila/drop 

0.5 

1 

-H 

brokan 

onfloor  A  brokan  A  full 

laak 

laak 

1 

1— - 

->full 

Given  C  —  (START  i  0,  drop  1 1),  the  following  is  a  closed  scenario  schema: 

Hi  =  ({fragile,  full}  t  START  10, 

{broken,  onfloor,  full}  1  drop!  1,  (1) 

{-'full}  {leak  1 12 ) 

where  I  have  written  partial  states  as  sets  of  literals,  all  other  propositions  being  unconstrained.  In  this 
example,  the  closedness  of  the  schema  depends  on  every  literal  except  the  last  occurrence  of  ->full. 
That  is,  if  any  other  partial  state  were  to  be  deconstrained,  the  schema  would  no  longer  be  closed  at  the 
next  position. 

Closed  scenario  schemas  have  the  useful  property  that  the  probability  of  the  set  of  scenarios  represented 
by  a  schema  is  not  infinitesimal,  as  the  following  theorem  will  show. 


Lemma  7  If  T  is  a  consistent,  simple  theory,  and  if  [H]^  is  a  discretized  closed  T,C,  L  scenario  for  a 
finite  occurrence  sequence  C,  then  then  for  any  L./i-model  of  T,C,  the  measure  assigned  to  the  set  of 
L,n- series  described  by  H,  which  I’ll  denote  by  _Mm([H]m),  is  given  by  <2„([E],4, 1),  where  is  defined 
by  the  following  recursive  construction: 

Let  [H],,  =  T  =  7i . 7 *.  If  j  is  a  position  in  T,  let  Ij  be  inextconat(j),  where  neztconst(j)  is  the 

subscript  of  the  first  constant  i  in  T  that  does  not  precede  7 j ,  that  is,  of  the  next  i  after  7,  of  the  form 
s  l  E  l  constant.  (If  jj  =  8j  {  Ej  J.  ij  is  itself  fixed,  then  neztconsi(j)  =  j  and  Ij  —  ij.  If  7,  is  variable, 
and  j  is  the  last  position  in  T,  then  let  Ij  =  \L/ji\.)  Define  1 r;  to  be  M,,(sj@i  +  1  |  A  Ej  \  i). 

This  quantity  does  not  depend  on  j,  because  constrains  enough  propositions  to  make  past  times 
irrelevant.  (Take  «o  =  F.)  Let  A  |  s  and  \e,  |  s  be  the  A  and  A e,  defined  in  clause  3  of  Definition  13, 
given  partial  state  s.  Once  again,  we  invoke  the  closedness  of  H  to  define  these  quantities.  If  Ej  =  <j>, 
then  \Bj  |  s  -  (j  -  A  |  s). 

Finally,  we  can  give  the  definition  of  Q The  base  case  is  Q^((),j)  =  (1  —  A  |  1  —  1 . 

(Take  »'0  =  —  1.) 

The  recursive  case  is  where  T  =  7^ ,  T'.  Then 

'  (1  -  (A  |  Sj-i)/!)';-*;-1-1  TjQ; 

. . . if  7;  =  Sj  1  Ej  i  f°r  some  Ej  [tj  €  C 

(1  -  (A  |  Sj-i)fl)r’~i’-t~1(\E,  |  S>-l)/UTjQ'M 
j)  _  ,  . .  .if  £j  is  fixed  and  not  derived  from  C 

h- 1 

otherwise 


broken 

f rag 11*  on floor 


0  10  20  30 


Figure  4:  A  Discretized  Closed  Scenario  Schema 

where  Q ^  =  Q„{T' ,j  +  1),  and  where  ij _ i  is  understood  as  a  variable  bound  in  the  for  Q<1(. . . ,  j  —  1) 
if  7j_i  is  variable  and  j  >  1,  and  as  the  constant  ij  _  i  if  7j_i  is  fixed  or  j  =  1. 

Proof:  This  formula  may  look  forbidding,  but  it’s  basically  just  repeated  application  of  Definition  13. 
Each  of  the  three  clauses  breaks  into  four  parts: 

Q,(rj)  = 

M ^(nothing  happens  from  ij _ i  +  1  to  ij)  x  M„(Ej  happens  at  ij) 
becomes  true  at  ij  +  1)  x  M^the  rest  ofT  happens) 

The  probability  of  nothing  (i.e.,  <f>)  happening  for  l  time  steps  is  (1  -  A/i)',  where  /  is  the  number  of  steps 
between  £j_i  and  Ej.  In  the  first  two  cases,  l  —  Ij  —  ij_i  -  1.  In  the  last  case,  Ej  can  go  anywhere 
between  ij-i  and  Ij ,  so  we  have  a  that  binds  ij  to  every  possible  location  for  Ej .  The  probability  that 
Ej  happens  after  time  ij  is  (A Ej  I  unless  Ej  is  drawn  from  C,  when  it’s  1.  The  probability  that 

becomes  true  after  ij  is  Jr;.  The  probability  that  the  rest  of  T  happens  is  Q'^.  In  the  base  case,  Q?((),j) 
is  the  probability  that  nothing  happens  after  the  j  -  l’st  event,  which  is  just  (A*  |  .  QED 

Aa  example  will  clarify  Lemma 7.  Consider  the  closed  schema  Hi,  discretized  with  p  =  0.1  (take  L  =  3): 

Ti  =  [Ei]o.i  =  {{fragile,  full}  }  START}  0, 

{broken,  onf  loor,  full}  }  drop  j  10, 

{-■full}  }  leak }  i2) 

and  let  «i  =  {fragile,  full},  s2  =  {broken,  onf  loor,  full},  S3  =  {-ifull}.  This  schema  is  illustrated 
in  Figure  4.  The  7’s  defined  in  Lemma  7  are  defined  thus:  I\  =  0, 12  =  10, 13  =  30.  Abbreviate 
1  —  (A  |  the  probability  that  nothing  happens  in  one  time  step  where  is  true,  as  pi.  p\  =  l,pj  = 
1  —  1  x  p  =  0.9,  p3  =  1.  Then 

=  Ad^(«i@l  |  START } 0)pi  x  A4„(s2@H  I  drop}  10  A  si @11) 

29 

x  Y  P2*-11(Aleak  I  s2)  *  AM»3@«2  +  1  |  leak}  »2)p3°_,J_1 
>3=11 

29 

=  0.15  x  1  X  0.5  X  Y  (0.9)<,_11  X  0.1  X  1 

>3=11 

=  0.15x0.5x0.86 
=  0.065 
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Lemma  7  does  not  specify  a  probability  for  ,  the  schema  obtained  from  Hi  by  deconstraining  fragile 
after  START.  But  intuitively,  that  probability  should  be  the  sum  of  the  probability  of  Hi  plus  the  prob¬ 
ability  of  the  schema  obtained  by  setting  fragile  to  #F  after  START.  The  theorem  I  will  prove  next 
formalizes  this  intuition,  by  showing  that  any  set  of  worlds  described  by  a  schema  can  be  reduced  to  a 
set  described  by  closed  scenario  schemas. 


Definition  21  A  scenario  schema  H'  extends  scenario  schema  H  if  they  have  the  same  Ef s  and  tj'a, 
and  each  partial  state  s'  in  H'  extends  the  corresponding  Sj  from  H. 


Definition  22  The  closure  of  a  scenario  schema  H  is  a  maximal  set  of  closed  minimal  extensions  H'  of 
H  (that  is,  each  H'  extends  the  states  of  H  just  far  enough  to  make  the  resulting  schema  closed),  such 
that  for  n  sufficiently  small,  each  [H']p  has  nonzero  probability  as  specified  in  Lemma  7, 


The  following  lemma  will  be  useful: 


Lemma  8  Let  H  be  a  T,C,  L-scenario  schema  of  length  I  for  a  consistent,  simple  theory  T,  and  let  5 
be  an  infinite  set  of  rule  instances  such  that  for  each  rule  instance  R  in  5  there  is  a  closed  schema  H' 
that  extends  H  and  has  nonzero  probability,  and  in  which  R  is  Arable  in  s'f  (where  =  s'j  [  . . .).  (5  is 
either  a  set  of  *|  rule  instances,  in  which  case  they  refer  to  the  probability  of  various  candidate  E/+is, 
or  a  set  of  _<— |  and  I—*,  rule  instances,  in  which  case  they  must  all  mention  the  same  E/+ j.) 

Then  there  is  an  arbitarily  large  finite  subset  of  S  such  that  there  is  a  closed  extension  of  H  in  which 
that  subset  is  Arable. 


Proof:  Because  there  are  an  infinite  number  of  extensions  of  H,  and  each  extension  mentions  only 
a  finite  series  of  events,  there  must  be  a  partial  state  sequence  s(,,...,Sj  such  that  an  infinite  num¬ 
ber  of  H's  contain  this  sequence,  and  there  are  an  infinite  number  of  distinct  extensions  of  the  form 
s'0, . . .  ,8'k,s'k+l, . .  .,s'r.  (We  take  so  to  be  F,  the  initial  state  with  every  proposition  false.)  In  other 
words,  if  you  view  the  extensions  as  a  tree  of  schemas,  there  must  be  a  node  of  infinite  degree  (by  Konig’s 
lemma).  There  may  be  more  than  one  such  partial-state  sequence,  but  we  can  let  k  be  the  position  of 
one  of  them.  Of  course,  k  could  be  0,  meaning  that  the  extensions  branch  infinitely  at  the  outset.  Label 
the  alternative  continuations  from  that  point  s'jQ l ,  s'^x , . . .,  and  for  each  R„  6  5,  let  be  the  partial 
state  in  which  Ru  is  a  firable  rule  instance.  See  Figure  5. 


Consider  a  particular  s^+j.  In  sjfejj  there  is  a  set  of  propositions  constrained  to  lie  in  {#T,  #F}  such  that 

not  every  other  constrains  them  the  same  way,  and  whose  truth  values  are  such  as  to  guarantee 
that  rule  instance  Rv  in  S  is  firable  with  nonzero  probability  in  the  corresponding  s).  Call  this  the 
support  for  Ru  in  s^, ,  and  give  it  the  name  T„.  This  set  is  finite,  because  a  given  s'j^j  differs  from  a 
given  only  in  which  of  a  finite  set  of  rules  fired  in  one  as  opposed  to  the  other.  It  is  possible  that 
an  infinite  subset  must  be  constrained  to  fire  or  not  fire  to  bring  about  ,  but  all  but  a  finite  number 


must  have  rj  =  1,  or  the  branch  containing  s'^x  would  have  zero  probability.  (This  is  where  we  use 
the  fact  that  the  theory  is  simple.)  In  such  a  case,  all  the  rule  instances  with  rj  =  1  would  fire  in  the 
generation  of  each  alternative  s'J^ ,  and  would  not  be  included  in  T„,  which  is  the  set  that  are  specific 


to  8 


,'M 
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Figure  5:  Schemas  Arranged  as  a  Tree  with  Infinite  Branch  Highlighted 


Choose  a  particular  rule  instance  €  S,  and  call  its  support  T\.  Because  T\  is  distinguished  from  its 
siblings  only  by  the  action  of  a  finite  set  of  rule  instances,  the  presence  of  7\  in  is  compatible  with 
all  but  a  finite  number  of  sibling  Tu's.  So  we  can  pick  an  Ri  whose  support  T2  is  made  to  appear  by  the 
action  of  a  different  set  of  rule  instances,  and  generate  a  partial  state  in  which  both  T\  and  Tj  are 
present.  (This  new  partial  state  might  or  might  not  be  among  the  s’k+l ’s  that  we  already  have.)  Getting 
Ti  and  Tj  still  leaves  an  infinite  number  of  T„’s  achievable,  so  we  can  repeat  this  process  indefinitely, 

yielding  Sj^’j . ^  for  any  N,  and  a  scenario  schema  in  which  N  rules  are  firable  in  some  successor  of 

sf  2 . N].  QED 


Theorem  9  If  E  is  a  finite  T,  C-scenario  schema  for  a  consistent,  simple  theory  T,  then  E  has  a  unique 
closure,  and  that  closure  is  finite. 


Proof:  The  proof  is  by  induction  on  the  length  of  the  schema.  A  schema  of  length  zero  is  already  closed. 
Assume  the  theorem  is  true  for  schemas  of  length  I  or  less,  and  consider  a  schema  E  of  length  7  +  1, 


which  ends  with  s/+1  |  £/+i  1  ti+i-  It  will  be  convenient  to  have  E  closed  at  ail  positions  through  /, 
so  if  it  isn’t,  remove  the  /  +  I’st  element  of  E,  close  the  shortened  schema,  and  then  tack  the  /  +  I’s 
element  back  on  to  each  element  of  the  result.  Call  this  operation  closing  the  prefix  of  E.  The  result  of 
closing  the  prefix  is,  by  the  induction  hypothesis,  finite.  If  it  is  empty,  then  the  prefix  does  not  describe 
any  timelines,  and  therefore  neither  does  E,  so  we  are  done. 

Otherwise,  perform  the  following  sequence  of  operations  on  each  element  of  the  closure.  For  simplicity,  I 
will  use  the  letter  E  again  for  such  an  element,  because  all  we  have  done  is  close  E  at  positions  through  /. 
If  E  is  not  closed  at  /  + 1  with  respect  to  every  P  constrained  by  s/+i,  pick  a  P  for  which  it  is  not  closed. 

Find  the  set  5  of  all  rule  instances  of  the  form - <— |  £7+ 1  \  . . .  or  . . . /Ei+i  |— ♦ . . . .  that  mention  P 

in  their  consequents.  Pick  one  rule  instance,  and  let  A\  A  ...  A  An  be  the  conjunction  of  its  antecedent 
and  the  annihilation  of  its  consequent.  If  possible,  create  an  extension  S'  of  E  with  s'j(Ai)  =  #T  for 
each  literal  A,.  (If  I  =  0,  S'  will  =  E  unless  one  of  the  constraints  would  require  some  proposition  to 
have  value  #T,  in  which  case  E  has  no  extension.)  If  E'  exists,  use  the  induction  hypothesis  to  close 
its  prefix,  once  again  splitting  into  a  finite  number  of  schemas  that  are  all  closed  through  /.  We  then 
repeat  the  process  for  a  set  of  n  candidate  extensions,  in  each  of  which  s'i(Ai)  =  # F  for  some  i  6  [1 ,  n] . 
We  have  now  generated  n  +  1  closures  of  extensions  of  E.  Each  schema  in  any  closure  is  closed  at  /  +  1 
with  respect  to  P.  Those  in  the  first  closure  all  have  the  rule  firable;  those  in  the  remaining  closures, 
unfirable.  See  Figure  6(a). 

If  there  is  no  way  to  make  the  first  rule  firable,  then  either  E'  will  not  exist,  or  its  closure  will  be  empty. 
In  this  case,  take  E'  instead  to  be  the  schema  obtained  by  taking  the  prefix  of  E  up  to  position  /,  and 
adding  a  constraint  s'^-iAi  V  ...  V  -'A„)  =  # T  (Figure  6(b)).  Adding  this  constraint  is  in  accordance 
with  Principle  V  because  it  does  not  change  the  probability  of  s'j  or  E',  since  we  have  already  shown 
that  there  is  no  extension  of  S  that  makes  the  disjunction  false.  Hence  it  leaves  E'  dosed  through  /. 

Take  all  the  extensions  generated  for  the  first  rule  instance  (just  one  if  the  rule  cannot  fire),  and  for  each 
repeat  the  process  for  the  next  rule  instance  that  mentions  P  in  its  consequent.  Continue  in  this  fashion 
until  all  rules  have  been  processed.  The  result  is  a  tree  of  extensions  of  E  whose  leaves  are  all  dosed  at 
I  +  l  with  respect  to  P.  (Here  and  for  the  rest  of  the  proof,  if  a  branch  is  infinite,  define  the  “leaf’  at 
the  end  of  it  to  be  the  schema  obtained  by  taking  the  intersection  of  all  the  partial  states  along  it.)  See 
Figure  7. 

All  these  extensions  are  obtained  by  making  truth- value  assignments  to  propositions  at  positions  before 
/  +  1,  so  if  E'  is  a  “leaF’  extension  for  the  tree  for  P,  it  will  be  closed  at  position  /  +  I  with  respect  to 
all  the  propositions  E  was  closed  with  respect  to,  plus  P.  If  there  are  still  propositions  it  is  not  closed 
with  respect  to  at  position  /  +  1,  repeat  the  process  with  each  of  them. 

When  we  have  generated  a  bunch  of  extensions  that  are  closed  at  /  + 1  with  respect  to  every  proposition, 
we  do  it  again  with  all  the  *-|  rules. 

When  this  process  is  complete,  we  will  have  produced  a  big  tree  of  extensions  of  E.  This  tree  must  have 
a  finite  number  of  branches,  even  if  the  number  of  rule  instances  is  infinite,  as  can  be  seen  by  supposing 
that  it  has  an  infinite  set  of  branches.  Because  E  is  finite,  there  are  only  a  finite  number  of  propositions 
constrained  in  s/+i .  Each  of  these,  by  the  induction  hypothesis,  gives  rise  to  only  a  finite  closure  at  times 
before  /  +  1.  So  either  there  is  a  proposition  P  for  which  there  are  an  infinite  number  of  extensions,  in 
each  of  which  a  different  rule  asserting  P  or  ->P  is  firable;  or  there  are  an  infinite  number  of  extensions, 
in  each  of  which  a  different  >|  rule  is  firable.  In  the  first  case,  Lemma  8  says  that  in  some  extension  of 
E  more  than  one  rule  asserting  P  or  ->P  is  firable,  which  is  impossible  if  T  is  consistent.  In  the  second 
case,  Lemma  8  says  that  in  some  extension  of  E  an  arbitrarily  large  number  of  _  — ►  |  rules  are  firable,  and 
hence  A,  as  defined  in  Definition  13,  is  >  1//4,  no  matter  how  small  we  take  /i,  which  is  also  impossible 
if  T  is  consistent  and  simple. 
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Knit  A  4  A  4  -A  4  ..A  -A 

12  *  • 


27@/>. 


Fimbit 


Unfir  obit 


(a)  Rule  is  firable  in  some  extension 


-A  V  -A  V  ...  V  -A 
1  2  n 

(b)  Rule  is  unfirable  in  any  extension 


Figure  6:  Extending  Closure  to  Make  Rule  Firable  or  Unfirable 


This  construction  defines  a  finite  closure  for  5.  It  remains  to  show  that  it  is  unique.  At  each  step  a 
minimal  extension  was  made  to  determine  the  firability  of  a  particular  rule.  Suppose  the  rules  had  been 
taken  in  another  order.  Any  two  orderings  differ  by  only  a  finite  number  of  transpositions  of  adjacent 
rules,  so  suppose  Ri  was  considered  just  before  Ri,  and  picture  considering  Ri  before  Ri  instead.  If 
consideration  of  Ri  does  not  result  in  a  change  to  E',  the  extension  to  E  built  so  far  (that  is,  if  it  is  not 
firable  in  any  extension  of  E'),  then  clearly  we  could  consider  Ri  first  and  R\  still  wouldn’t  be  firable. 
If  R j  does  generate  one  or  more  nontrivial  extensions,  then  first  consider  the  one  in  which  /2j  becomes 
firable.  If  Ri  is  still  firable,  then  obviously  we  could  have  switched  the  order.  If  Ri  is  not  firable,  but 
would  have  been  if  we  considered  it  first,  then  there  must  be  a  literal  P  occurring  in  the  antecedent 
of  Ri  such  that  ->P  occurs  in  the  antecedent  of  Ri,  and  P  is  not  constrained  in  S'.  Hence  there  is  an 
alternative  extension  of  S'  in  which  Ri  is  not  firable  solely  because  P  is  constrained  to  be  false.  That 
extension  then  gets  further  extended  to  a  S"  in  which  Ri  is  firable.  If  Ri  had  been  considered  first,  S" 
would  have  been  generated  as  an  extension  of  E'  directly.  QED 

All  of  the  preceding  lemmas  are  consequences  of  Definition  13.  However,  I  have  not  shown  that  is 
actually  a  probability  measure  on  sets  of  L,  /4-series.  To  demonstrate  that,  we  first  extend  the  definition 
of  M». 
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(not  closed  at  l+l ) 


Rule  1  extensions: 


1  firable 


l  not  firable 


1  not  firable 


NO  EXTENSIONS 


2  not  firable 


Finite  set  of  "Leaves," 
all  closed  at  1+1 

Figure  7:  Tree  of  Extensions,  “Leaves”  Closed  at  /  +  1 


Definition  23  If  S  is  a  T,C,  L  scenario  schema,  define  the  measure  of  the  set  of  L,pt- series  described 
by  S  as 

]/j)  =  ^ 

Sc€CL(E) 

where  CL(S)  is  the  closure  of  E. 


Lemma  10  If  T  is  a  consistent  theory,  and  if  E  is  a  T,C,  L  scenario  schema,  P  is  a  proposition  symbol, 
and  j  is  a  position  in  E,  with  the  j’th  element  of  S,  =  sj  |  Ej  |  tj ,  where  Sj  does  not  constrain  P, 
then 


Af„([SU  =  -M„([E  +  PI  <;]„)  + AU[E  +  -Pi  <>]„) 


where  E  +  a  Itj  means  S  extended  to  constrain  a  to  be  true  in  the  Sj.  (If  there  is  no  such  extension, 
take  M,,( S  +  aft,)  =  0.) 


Proof:  By  induction  on  j,  the  position  where  the  extension  occurs.  The  theorem  is  obvious  for  j  =  0, 
because  «o  constrains  every  P  to  be  # F.  probability  of  P  being  true  in  s0  is  0.  Now  assume  it’s  true 
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through  position  j  —  1,  and  consider  position  j  >  0.  Let  E+  and  E_  be  the  extensions  for  P  and  ~>P 
respectively.  If  P  or  ->P  is  already  true  in  an  element  of  CL([E](,),  then  that  element  will  also  be  an 
element  of  C£([E+]p  and  C£([E_]M.  So  consider  an  element  Ee  that  does  not  take  a  position  on  P  at  j. 
The  closures  of  the  extensions  of  Ee  for  P  and  ->P  will  constrain  exactly  the  same  sets  of  propositions  in 
Sj- 1,  as  a  glance  at  Definition  13  will  show.  For  example,  in  some  closed  extensions,  P  will  be  false  at 
j  —  1  and  a  rule  causing  P  will  be  firable  with  probability  r.  The  Q M  clause  (see  Lemma  7)  for  in  the 
extension  for  P  will  differ  from  the  corresponding  clause  for  the  original  E  in  having  an  extra  factor  of 
r  in  computing  t,,  while  the  same  clause  for  the  extension  to  ->P  will  have  a  factor  of  1  —  r.  Similarly, 
but  in  the  opposite  sense,  for  extensions  where  P  is  true  at  j  -  1  and  a  rule  causing  ->P  is  firable.  In 
extensions  where  P  is  false  at  j  —  1  and  no  rule  asserting  it  is  firable,  we’ll  give  weight  0  to  P  being  true 
at  j,  and  weight  1  to  its  being  false;  and  symmetrically  in  the  opposite  case.  In  all  cases,  wherever  there 
is  a  closure  with  weight  p  for  P,  there  is  a  closure  with  weight  1  —  p  for  ->P.  Hence  we  can  combine  each 
of  these  pairs  and  rewrite 

Af„([E  +  P I  tjU  +  AM(E  +  -'P  l  tjU 

as  a  sum  of  closed  extensions  of  E,  each  constraining  a  finite  number  of  propositions  at  positions  j  and 
earlier  in  all  possible  ways  that  make  each  rule  relevant  to  P  firable  or  not  firable.  These  constraints 
are  mutually  exclusive  and  exhaustive,  so,  by  the  induction  hypothesis,  the  sum  of  Q ^  for  these  closed 
extensions  is  Q^([E]^).  QED 


Lemma  11  The  measure  assigned  to  the  set  of  all  £,/i  series  is  1;  that  is, 

n/ei 

«= o  s  es. 

where  Si  is  the  set  of  all  T,C,  £  scenario  schemas  of  length  i  each  of  whose  elements  is  of  the  form 
S  i  E 1 1  for  some  E. 


Proof:  Take  the  closures  of  each  scenario  schema,  and  ‘  </>-ify”  each  of  the  closed  schemas  obtained; 
that  is,  split  each  schema  into  all  the  possible  schemas  of  length  v  =  \L/fi\  obtained  by  intercalating 
occurrences  of  event  <f>.  Each  such  intercalation  requires  inserting,  after  a  triple  s  J  E  J  t,  zero  or  more 
triples  of  the  form  s  J.  4>  1 1,  with  exactly  the  same  s.  The  resulting  schemas  remain  closed,  because  <t> 
doesn’t  change  the  world  state.  In  addition,  inspection  of  Theorem  7  shows  that  if  a  closure  element  has 
probability  q,  then  the  sum  of  the  probabilities  of  the  schemas  derived  from  it  by  ^-ification  is  also  q. 

We  now  have  a  collection  of  closed  schemas,  all  of  length  u.  We  can  view  them  as  constituting  a  tree  if 
we  combine  prefixes  of  schemas  that  agree  on  a  sequence  of  events  and  states.  Before  building  this  tree, 
we  make  sure  that  if  two  schemas  agree  up  to  a  point  and  then  differ,  the  difference  is  either  that  they 
have  different  events  occurring  at  that  point,  or  they  have  disagreeing  partial  states.  We  eliminate  the 
case  where  one  schema  has  a  state  si  that  is  an  extension  of  the  corresponding  state  sj  from  another 
schema  by  invoking  Lemma  10  to  split  the  second  schema  into  as  many  extensions  as  are  necessary  to 
make  sure  that  all  either  disagree  with  si  or  are  identical  with  it.  Lemma  10  then  says  that  the  sum  of 
the  probabilities  assigned  to  the  resulting  schemas  is  equal  to  the  schema  that  contained  s^.  In  addition, 
each  split  schema  remains  closed,  because  the  distinctions  introduced  are  (a)  supported  by  preceding 
states;  (b)  irrelevant  to  later  ones  (else  the  closure  would  already  have  taken  a  stand  on  the  affected 
propositions). 

After  these  machinations,  we  can  view  our  collection  of  closed  schemas  as  a  tree  of  states  and  events. 
At  the  root  is  the  initial  state,  F.  Each  state  has  a  set  of  events  as  children,  and  each  event  has  a  set 
of  states  as  children.  The  children  of  an  event  E  are  all  nonzero-probability  states  that  can  result  from 
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E  given  that  E's  parent  is  the  state  obtaining  just  before  E,  and  that  can  be  obtained  by  combining 
propositions  relevant  to  the  occurrence  of  a  later  event.  The  children  of  a  state  are  all  the  events  that 
can  occur  in  the  next  /i-Iength  time  interval  if  that  state  obtains  just  before  it  (including  exogenous 
events  and  occurrences  of  <f>). 

It  is  now  a  matter  of  a  simple  inductive  argument  to  verify  that  at  each  the  conditional  probabilities  of 
the  children  given  by  Definition  13  sum  to  1.  QED 

These  two  lemmas  have  a  simple  corollary 


Theorem  12  is  a  probability  measure 


Proof:  To  be  a  probability  measure,  Mu  must  assign  measure  0  to  the  empty  set,  1  to  the  of  all  L, mi¬ 
series,  and  must  assign  p\  +P2  to  U  Wi  if  it  assigns  p\  to  Wi  and  pi  to  W^.  The  preceding  lemmas 
allow  us  to  conclude  that  it  does.  QED 

Now  (at  last!)  we  can  show  what  happens  as  we  move  to  the  continuous  case. 


Theorem  13  If  a  theory  T  is  consistent,  then  for  any  exogenous  occurrence  sequence  C,  and  any 
T,  C,  L-scenario  schema  E  with  p  fixed  elements  not  derived  from  C, 

^  1 

M(E)  =  lim »  —  .VUPU 

H  —  0  pF 

exists  and  equals 

£  Q( 

Hc€C£(  E) 

where  Q  is  defined  recursively  in  a  way  analogously  to  Q^,  as  follows.  Let  7}  be  tneztconst{j)  where 
neitconst(j)  is  as  defined  in  Lemma  7.  (If  =  Ej  [tj  is  fixed,  then  Tj  —  tj.  If  ^  is  variable,  and  j  is 
the  last  position  in  E,  then  let  7)  =  L.)  Define  n}  ,A  |  s.  and  A e,  |  s  as  in  Lemma  7. 


The  base  case  is  now 

Q(  (),;)  = 

where  we  take  s o  =  F  and  t0  =  0.  The  recursive  case  is  for  E  =  ,  E',  when 

j-i  Q1 

...  if  ^  =  Sj  j  Ej  j  tj  for  some  Ej  i  tj  €  C 
e-(A|»,_,)(T,-«,_i)(A£j  |  0j-\)irjQ' 

Q{S,j)  =  ^  . .  .if  £j  is  derived  from  a 

T 

f  ’  e-W «,-iX*,-«j-.)(ab.  |  Oj.^jQ'dtj 

otherwise 


where  Q'  =  Q(5',  j  +  1),  and  where  tj- \  is  understood  as  a  variable  bound  in  the  /  for  Q(.  .  ,j  —  1)  if 
£j- 1  is  variable  and  j  >  1,  and  as  the  constant  tj- 1  if  £j-i  is  fixed  or  j  =  1. 
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Proof:  First,  I  prove  that  for  a  given  closed  scenario  schema  Ec,  limM_o  ^A4M([EC]M)  =  Q(EC)  1)  This 
fact  follows  from  the  fact  that 


lim(l  -  =  e"A(,J"‘l) 

p—0 

when  v  is  a  constant.  This  formula  applies  directly  to  the  derivation  of  the  clauses  of  Q’s  definition  from 
the  corresponding  clauses  of  QM.  In  the  second  clause,  the  rrce  p  —  0,  but  there  are  p  such  factors  in 
the  overall  definition  of  (J(E,  1),  and  they  are  canceled  out  by  the  l/pp  factor  before  the  ]TV  In  the  third 
clause,  p  becomes  a  bound  dtj,  over  which  the  integration  is  performed. 


So 


2*-Uo  pp  £ 

He6Ct(  =  )  ^ 

E  Q^-1) 

—  /-  r  /  “  » 


QED 

I  can  now  define  the  probabilities  of  propositions  in  terms  of  the  probabilities  of  scenario  schemas. 


Definition  24  If  C  is  a  sequence  of  exogenous  events,  and  q  is  a  conjunction  of  random  variables  of  the 
form  described  in  definition  15,  then  let  55c(a)  be  the  set  of  all  scenario  schemas  C-compatible  with  a. 
A  scenario  schema  E  is  C-compatxble  with  a  if  and  only  if 

1.  for  each  E [  t  in  C  or  a,  there  is  an  element  s  j  E  j  t  G  E  for  some  s; 

2.  for  each  -<E'  it  in  a,  there  is  an  element  s  l  E  j  t  E  H  for  some  s  and  some  E  £  QU  E  ^  E'; 

3.  for  each  t  mentioned  in  a,  there  is  an  element  s  i  E 1 1  £  E  for  some  s  and  some  E  €  Q  U  {4>} ; 

4.  for  each  A  i  t  in  a  there  is  an  element  s  1  E 1 1  for  some  E  £  Q,  such  that  s(A)  =  # T; 

5.  for  each  Aft  in  a  there  are  two  successive  elements  si  1  E\  j  ti,  s^i  E2 1  tj  £  E  such  that  t2  =  t  and 
«i(A)  =  #T  (unless  t  =  0,  in  which  case  E  is  compatible  with  a  only  if  A  is  a  negated  proposition 
symbol); 

6.  and  for  each  triple  s  )  E 1 1  in  S,  s  is  not  constrained  except  as  stipulated  above. 

In  general,  55(a)  will  be  infinite,  on  two  counts:  First,  there  can  be  an  infinite  number  of  event  sequences 
that  make  a  true;  second,  if  A  1 1  or  -<E'  1 1  occurs  in  a,  and  Q  is  infinite,  then  55(a)  will  have  to 
include  an  element  s|5|f  for  every  E  £  Q  ^  E'.  However,  in  the  case  of  infinities  of  the  second  kind, 
most  of  the  schemas  will  have  probability  0,  assuming  the  theory  is  consistent. 


Definition  25  If  0  is  an  arbitrary  boolean  combination  of  random  variables  of  the  form  described  in 
definition  15,  then  express  0  as  a  disjunction  of  mutually  exclusive  conjunctions  of  random  variables 
ai  V  ...  V  a*,  and  define  M(0)  to  be 


E  £ 


or.  E€SS(a,) 
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Now  all  I  have  to  do  is  show  that  the  clauses  of  Definition  5  actually  follow  from  this  definition  of  M. 


Theorem  14  Let  T  be  a  consistent,  simple  theory,  C  be  an  occurrence  sequence  of  length  n,  and  L  be 
a  real  number  >  duraiion(C).  If  W  is  the  set  of  worlds  obtained  as  the  limit  of  W„  as  p  — *  0,  and  M  is 
as  defined  in  Theorem  13,  then  M  satisfies  all  the  constraints  in  Definition  5. 


Proof:  I’ll  go  through  the  clauses  in  order. 


1.  For  any  proposition  A  €  V,  A<(.AtO)  =  0.  This  is  a  trivial  consequence  of  Definition  25. 

r  r 

2.  If  T  contains  a  rule  instance  A/E  |— *_  B  or  a  rule  instance  B  _«— |  E\A,  then  for  every  date  t, 
require  that,  for  all  nonempty  conjunctions  C  of  literals  from  B: 

M(C[t  |  E  It  A  A]t  A~B  \  t)  =  r 


To  show  this  clause,  start  with  this  fact: 


M(C\t  |  EitAA]tAB]t)  = 


M(CjtAEjtAA\tAB]t) 
M(Elt/\A]t  AB]t) 


Both  the  numerator  and  denominator  are  defined  as  in  Definition  25,  and  hence  each  will  be  a  sum 
of  M  values  over  a  set  of  scenario  schemas.  Because  they  mention  the  same  times,  the  numerator 
and  denominator  will  sum  over  the  same  scenario  schemas,  except  that  those  for  the  numerator 
will  have  partial  states  [  E 1 1  that  constrain  C  to  be  #T.  When  the_schemas  are  closed,  the  two  sets 
retain  the  same  relationship,  because  any  closure  that  has  A]t  A  B]t  has  a  rule  instance  for  C  [t 
that  is  firable.  Hence  the  denominator  will  be  a  sum  over  a  bunch  of  closed  schemas  with  E[t, 
and  the  numerator  will  have  a  similar  bunch,  but  with  s  l  E  It,  where  s(C)  =  #T.  Consulting 
Definition  13,  we  see  that  the  ratio  of  the  two  partial  states’  probabilities  is  r.  This  ratio  will 
get  reflected  as  the  ratios  of  the  corresponding  jt/s  in  the  Q  for  each  schema  (cf.  Lemma  7),  and 
therefore  we  can  pull  the  r’s  out: 


_  r  Use cl(s)  M(=.) 

H=ecL(S)  ^(“) 

where  S  =  the  union  of  the  closures  of  the  schemas  for  £’|.tA,4TtABtt. 

3.  Suppose  B  is  an  atomic  formula,  and  let  Ri ,  Ri , ...  be  the  set  of  all  instances  of  I—*-  or  _«— |  rules 

r, 

whose  consequents  contain  B  or  ->B.  (The  set  may  or  may  not  be  infinite.)  If  Ri  =  A,/ Ei  |— ►_  C< 

r,- 

or  Ci  |  Ei\Ai,  then  let 

Di  =  Ai  A  Cl 

(The  Di  will  be  identically  false,  in  the  case  where  R  is  empty.)  Then 

M(B[t  \  B]tA  N)  =  1 
M{B[t  \-<B]t  A  N)  =  0 

where 

N  =  (-’£'1 1 1  V  —>D\  1 1) 

A(-<E 2  1 1  V  -'£>2  T  0 
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To  prove  thise  clause,  we  have  to  rewrite  N  as  a  disjunction  of  mutually  exclusive  conjunctions. 
At  most  one  of  the  Ei  can  happen,  so  we  can  start  by  trying  to  express  N  in  the  form: 

(~'E\  j  t  A  j  t  A  . . .) 

V  (Fi  A  F\\) 

V  (EiltAFn) 

V  ... 

V  (F2.UAF21) 

V  ... 

where  Fij  is  the  j’th  conjunction  that  prevents  any  rule  for  E,  from  firing.  There  are  duplicates 
among  the  Ei,  so  for  a  given  i  we  produce  the  list  Fn, ... ,  F,j,  by  finding  conjunctions  of  literals i t 
that  (a)  mention  every  proposition  found  in  a  rule  for  F,;  and  (b)  negate  at  least  one  literal  in 
each  rule  for  Ei.  The  resulting  Fij  for  a  given  Ei  are  mutually  exclusive,  so  each  conjunct  of  the 
disjunction  corresponds  to  a  separate  scenario  schema.  That  is,  either  no  Ei  occurs,  or  exactly  one 
Ei  occurs  plus  one  of  the  2**  —  1  mutually  exclusive  ways  in  which  its  £),  can  be  false.  For  each  of 
these  possibilities,  create  a  scenario  schema,  and  find  its  closure.  No  rule  for  any  Ei  is  Arable  in 
any  element  of  the  closure,  so  according  to  Definition  13,  the  probability  is  1  that  keeps 

the  value  it  has  —  1,  and  this  property  will  be  preserved  in  the  limit. 

4.  For  every  time  point  t  such  that  no  occurrence  with  date  t  is  in  C,  and  for  every  event  type  E,  let 
R  be  the  set  of  all  rule  instances  of  the  form  E  in  T,  and  suppose  that  S  is  an  arbitrary 

dj 

subset  of  R,  and,  letting  rj  =  Aj  »|  F,  define 

a  =  (  A  A (  V 

r,€S  r,eR-S 

and  As  =  £r>es  1  /dj.  Then  if  M(A  1 1)  ^  0,  require  that 

M{  some  occurrence  of  E  between  t  and  t  +  dt  |  A  T  t)  =  As 
(We  have  dropped  the  dt  in  favor  of  a  convention  that  M  is  a  density.) 

To  prove  this  clause,  express  A  as  a  possibly  infinite  disjunction  of  possibly  infinite  mutually 
exclusive  conjunctions  of  random  variables  of  the  form  P\t  and  ->P  ]  t.  (As  in  the  previous  clause, 
include  in  each  disjunct  enough  literals  that  no  two  disjuncts  are  consistent).  Call  the  k' th  conjunt 
Bt .  By  Definition  25,  the  conditional  probability  may  be  expressed  as 

Sit  M(E  A  Bt  T  Q 
J2kM(Bk]t) 

The  scenario  schemas  for  the  ifc’th  element  of  the  denominator  will  be  the  union 

U  5t£' 

£'€Cu{*}-{£} 

where  S  k,E'  is  the  scenario  schema  for  B*  f  t  that  has  the  element  S  l  E'  It.  (Cf.  Definition  24.) 
The  closure  of  Ek,E'  >s  exactly  the  same  as  that  for  E *,£,  except  for  the  switch  from  E'  to  E.  Hence, 
when  the  M  in  the  denominator  is  expanded  into  a  sum  over  the  closures  of  scenario  schemas,  we 
can  group  the  terms  of  the  sum  into  groups  that  differ  only  by  having  a  factor  A e'  in  their  Q’a. 
These  all  sum  to  1,  because  the  probability  of  <f>  is  defined  as  1  -  ^ B '  (see  ^fini^00  13). 

Hence  the  fraction  above  will  be  analyzed  as  the  limit  of 

A sP  +  A sn  •+  ■  ■  ■  +  A s/i 

/i  +  H  +  ...  +  fi 

as  fi  — *  0.  In  this  fraction,  the  numerator  and  denominator  have  the  same  number  of  terms,  a 
number  that  increases  as  fi  —*  0.  The  limit  of  the  fraction  is  A 5. 
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5.  If  one  of  the  previous  clauses  defines  a  conditional  probability  M(a  |  /?),  which  mention  times  t, 
then  a  is  conditionally  independent,  given  a,  of  all  other  random  variables  mentioning  times  on  or 
before  t.  That  is,  for  an  arbitrary  7  mentioning  times  on  or  before  t,  M(a  |  0  A  7)  =  M(a  |  /?). 
This  follows  from  clauses  4,  2,  and  3  of  Definition  13. 


I  will  end  this  appendix  with  a  couple  of  observations  about  what  we  have  proved.  Theorems  9  and  14 
imply  that  a  given  consistent,  simple  theory  has  exactly  one  model.  This  situation  is  quite  different 
from  that  of  first-order  logic,  where  most  consistent  theories  have  tin  infinite  number  of  models.  The 
reason  for  the  difference  is  the  heavy  hand  of  the  closed-world  assumption  (Reiter  1978)  applied  to  the 
initial  state  (where  all  propositions  are  taken  to  be  #F).  Any  uncertainty  in  the  system  must  be  modeled 
in  terms  of  probabilities.  So  you  are  allowed  to  say  that  the  probability  is  0.01  that  your  car  has  been 
stolen  today,  but  not  to  say  that  you  simply  don’t  know  whether  it  has  been  stolen  or  not.  If  we  relaxed 
the  closed-world  assumption,  multiple  models  would  reappear 

Another  observation  is  that  the  results  here  apply  only  when  the  timeline  system  is  left  to  its  own 
devices,  i.e.,  when  a  fixed  list  of  exogenous  events  is  picked  in  advance,  and  the  question  is  how  the 
world  will  react  to  those  events.  In  practice,  the  planner  uses  the  projection  system  this  way  only  to 
simulate  short  intervals  over  which  the  agent  is  refraining  from  action.  When  a  reactive  plan  is  projected, 
the  agent’s  next  actions  (i.e.,  the  next  exogenous  event)  is  influenced  by  the  state  of  the  timeline  just 
before  that  action,  so  the  theorems  proved  here  do  not  apply.  For  example,  in  the  “Little  Nell”  problem 
(McDermott  1982),  we  wish  to  project  the  chances  that  our  agent  will  be  able  to  protect  Nell,  who  is 
tied  to  the  tracks,  from  being  crushed  by  a  train.  The  timeline  manager  alone  might  give  us  a  high 
probability  of  disaster,  even  if  the  agent  is  allowed  to  choose  an  arbitrary  series  of  actions  in  advance. 
However,  if  the  agent  is  allowed  to  base  its  actions  on,  e  g.,  where  it  projects  it  will  find  Nell,  then  the 
probability  of  a  happy  ending  will  be  higher. 
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